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ABSTRACT 

This monograph describes 30 college and university 
programs funded by the Fund for the Improvement of Postsecondary 
Education from 1989 to 1991. Each description includes information on 
program purpose, project activities, major insights and lessons, 
project continuation, and available information. The first group of 
10 are programs focused on assessment and include an assessment 
resource center, area concentration achievement testing with 
curricular evaluation, computers and college writing, assessment 
seminars, New Pathway Curriculum impact evaluation, liberal education 
model assessment, college-wide measures toward general education 
goals, comprehensive assessment in academic disciplines, and a 
regional assessment network. Another group of four programs address 
college teaching: professional development, medical scholars, and 
database and online service orientation. Nine projects address 
curriculum and teaching in the disciplines including laboratory 
education, undergraduate mathematics, economic curricula, scientific 
thinking, French language and culture, case study physics, music 
theory, biology instruction, and freshman chemistry. Two programs 
address general education. Three projects involve teacher education 
and two programs address ethics instruction. The following 
institutions are included: University of Tennessee; Austin Peay State 
University (Tennessee); City University of New York; Harvard 
University (Massachusetts); Miami University (Florida); State 
University of New York; Winthrop College (South Carolina); University 
of California; Ohio State University; Salem State College 
(Massachusetts); Clemson University (South Carolina); Denison 
University (Ohio); Dickinson College (Pennsylvania); Tufts University 
(Massachusetts); University of Maryland; New Mexico State University; 
Northwestern University (Illinois); University of Oregon; University 
of Rhode Island; University of North Texas; Indiana University of 
Pennsylvania; Northern Virginia Community College; Union College (New 
York); University of Connecticut; and Saint Cloud State University 
(Minnesota) • (JB) 
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PREFACE 

The Fund for the Improvement of Postsecondary Education--]? ±PSE 
to its friends-supports pilot tests of new ideas in postsecondary 
education. This volume describes some interesting campus experi- 
ments whose support from FIPSE ended between 1989 and 1991. In 
addition, the volume attempts to draw usable conclusions, asking what 
worked, what didn't work, and why. It has been designed to help 
American colleges and universities facing issues similar to those that 
faced the directors of these projects. Among the issues: 

o how to assess student learning objectively while protecting 
faculty ownership of the assessment process; 

o how to use computers in college teaching; 

o how to broaden professional education; 

o how tc raise minority-and majority-achievement in math and 
science; 

o how to integrate teacher education into a liberal arts 
curriculum. 

This book is the second in a series. In October of 1990, when the 
first volume of Lessons Learned from FIPSE Projects appeared, the 
entries on the various projects were described as project self-portraits 
polished by FIPSE staff. The same applies this time around. Of 
course that is not to say that FIPSE staff helped the project directors 
paint their work in favorable colors. It means that while the entries 
rest ultimately on outcomes data supplied by the project directors 
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themselves, FIPSE staff helped to elicit important facts from the 
project directors by posing hard questions, and FIPSE staff drafted the 
entries. The leader of this rather complex undertaking has been Dr. 
Dora Marcus, FIPSE's specialist in project evaluation. 

But the differences between this monograph and its predecessor are 
significant. This volume harvests the first generation of FIPSE 
projects which had been consciously designed from the outset to be 
evaluable, FIPSE having boosted its commitment to evaluation in 
1986. Hence the following accounts of what did and did not work are 
more thorough than some of those in Lessons Learned I . A second 
difference is the number of projects described-30 rather than 15, with 
especially strong groupings in the areas of assessment and computer 
assisted instruction. 

The temptation to draw lessons from the process of drawing lessons 
about postsecondary education must be resisted. (What would come 
next, after all?) Still, several things strike an observer. First, it is 
notable how powerful good outcomes data can be in selling other 
practitioners on a given innovation, once those practitioners are 
persuaded of a need. Rhetoric may be the key to raising consciousness 
about a certain academic problem or issue, but it is the innovator 
armed with figures who wins support for a particular solution. On the 
other side, however, it is puzzling how slow many postsecondary 
educators are to compile evidence about what works in teaching and 
learning, considering the stress they typically lay on evidence in their 
scholarly work. 

Last, in the midst of this case for drawing and sharing the lessons 
of reform, a note of caution. Paradoxically, it might be wise to 
reinvent some kinds of wheels on every campus, rather than learning 
directly from the achievements of others. One thinks, for example, of 

ii 




general education curricula. Local wrangling about the content of the 
general education curriculum may play an important role in the pro- 
fessional development of the academic community, and the resulting 
sense of local ownership may be crucial in motivating faculty to take 
on what are often seen as less glamorous teaching assignments. That 
the particular content of the general education curriculum at a campus 
may in fact have little impact on the kinds of intellectual growth most 
people care about (see below, p.38) makes this seem less paradoxical. 

The project-portraits that follow are of course mere sketches. But 
all of our former project directors have indicated their willingness to 
fill in details, and they will welcome calls from readers of this mono- 
graph who have been intrigued or inspired by what they read here. 

Charles H. Karelis, Director 

Fund for the Improvement of Postsecondary Education 
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ASSESSMENT 



FIPSE has had an interest in assessment from the time of its 
founding. Of the 89 projects the Fund supported in its first round of 
grants in 1972, 15 concerned assessment of student learning outcomes. 
By 1990, 29 Comprehensive Program projects involved assessment. 

The ten successful programs described here range from large inter- 
national conferences to particular projects in individual institutions. 
The prevalence of consortial projects and other kinds of large scale 
activities in this area provides a notable contrast with other program 
areas, where individual institutional activity is the rule. This 
phenomenon can be attributed to a variety of causes, including pres- 
sures from state legislatures and regional accrediting agencies, the 
manifest advantages of pooling experience in an area where strategy 
and technique are just beginning to develop, and the need to have a 
larger number of students with whom to validate outcome measure- 
ments than a single institution can provide. 

The last two of these problems becomes particularly acute in the 
case of particular disciplines. Here even consortial arrangements, 
while solving the problem of having a sufficiently large pool of 
students, run up againjt the problem cf differences of approach and 
emphasis from one participating department to another. 

These programs illustrate a rich variety of assessment strategies 
and instruments. Departing from early efforts that relied on input 
factors, nationally normed tests and quantifiable outcomes like grad- 
uation rates and alumni satisfaction, the new assessment programs 
look at specific performance outcomes. Tests are likely to be faculty- 
made and tailored to measure success in achieving precisely defined 
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learning goals. Comparative assessments are well controlled, use 
statistics carefully and employ a variety of assessment strategies. 

For all the technical care that has gone into developing these 
assessment projects, conclusive demonstration of the degree and full 
extent of student learning as a result of a specific activity remains 
elusive. At present, it is not possible to measure all aspect? of student 
learning, since only a few instruments can demonstrate differences 
that are both conclusive and substantial. The problem is compounded 
ty the cumbersome logistics of following students for long enough to 
assess the full effects of their educational experiences and ensuring 
their participation in assessment activities in which they have no 
personal stake. 

The proliferation of assessment programs further suggests a need, 
not directly addressed by these projects, to develop ways to assess 
assessment. As outcome assessments become a common feature of the 
higher education landscape, FIPSE will expect to see more effort in 
this direction. 
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BUTTRESSING ASSESSMENT 



AAHE's Assessment Forum 
The University of Tennessee, Knoxville's Assessment 
Resource Center 

Purpose 

As the assessment movement gathered momentum a decade ago, it 
became apparent that colleges, universities, and academic departments 
across the nation would need help learning the rudiments of assess- 
ment and making plans to evaluate their programs. Since there exist- 
ed no resource centers dedicated to assessment, and no bibliographies 
or directories, institutions that had emerged as leaders in the field, 
such as Alverno College, Northeast Missouri State University, Miami- 
Dade Community College and the University of Tennessee, Knoxville, 
were overwhelmed with requests for guidance and information. 

In 1986 FIPSE funded two projects designed to share expertise and 
resources on assessment with national and international audiences, 
and to foster and publicize advances in the field: the University of 
Tennessee, Knoxville's Assessment Resource Center, and the Assess- 
ment Forum of the American Association for Higher Education. Dur- 
ing the three years that followed, the men and women who staffed 
these projects answered thousands of inquiries, presented and pub- 
lished scores of papers and reports, and organized conferences and 
workshops whose principal problem became how to accommodate the 
escalating numbers of participants. These efforts to inform, assist, and 
guide the dialogue on teaching and learning have helped to revitalize 
undergraduate education on many campuses in recent years. 



Project Activities 



From the 1990's vantage point, it is difficult to envision the 
assessment field, and indeed higher education, without AAHE's 
Assessment Forum and The University of Tennessee, Knoxville's 
(UTK) Assessment Resource Center. In the mid-1980's, however, 
colleges and universities were struggling in isolation with state 
accountability mandates, accrediting agency requirements, and internal 
concerns with declining student performance and the efficacy of the 
curriculum. The only help available came from word of mouth, 
impromptu exchanges, and serendipitous discoveries in the library. 

Thus, the logical beginning for both AAHE and UTK was to gather 
in a systematic fashion the information on assessment that was 
dispersed throughout the country. Accordingly, the staff of the 
Assessment Forum and the Assessment Resource Center set about 
commissioning and writing papers, visiting campuses, and organizing 
workshops to learn about and contribute to efforts across the nation. 

The volume of the dissemination activities undertaken by these 
projects explains their emergence as major forces in assessment. Over 
a three-year period, UTK's Assessment Resource Center produced 
eight major reference works, 19 research reports, an international 
conference, a consortium of campus-based leaders, a cross-national 
study of assessment, several assessment bibliographies, and a national 
newsletter- Assessment Update , published by Jossey-Bass, the first 
newsletter dedicated to assessment in postsecondary education. It 
provided services to campuses in 49 states plus D.C. and Puerto Rico 
and five foreign countries through workshops anr^ visits, and sent 
materials to 1400 colleges, universities and organizations. 
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Between 1986 and 1989 AAHE's Assessment Forum distributed 
approximately 30,000 resources to individuals and organizations. The 
Forum compiled available literature, commissioned and produced a 
series of papers (now in second printing) on various aspects of 
assessment, published a directory of outstanding assessment projects 
and organized annual conferences that drew close to 3,000 people. 
AAHE's own publications helped to spread the word. Two issues of 
the AAHE Bulletin focused on assessment, and Change magazine be- 
came a major vehicle for news and reflection on the field, especially 
the award-winning article in the October, 1990 issue on assessing 
assessment. 

Major Insights And Lessons Learned 

The work of these two centers was not simply dissemination, how- 
ever. Both were actively engaged in research and analysis as well, 
attempting to identify lessons and trends in assessment. Their 
findings include the following: 

o The workshops, readings, arguments and meditations of the last 
half decaie have not been in vain. The higher education communi- 
ty has grown increasingly sophisticated about assessment. For 
example, the audiences of the early AAHE assessment conferences 
manifested above all an urgent desire to understand assessment so 
as to ward off its anticipated dangers. Within a year or two, 
however, attendees began to express a need for guidance in their 
search for the best models. Eventually, based on their own ex- 
perience, faculty and administrators undertook a quest for alter- 
natives to standardized instruments. At present, there is a new 
sense of confidence with respect to assessment on the part of higher 
education professionals, and a concern with keeping assessment 
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connected to the classroom and making maximum use of its educa- 
tional potential. 

o The profession's preferences regarding instruments of assessment 
have also shifted. An early interest in standardized tests and 
external examiners has given way to exploration of alternative 
approaches such as self-assessment, portfolios, and interviews, 
brought on in part by an awareness of the diversity of institutional 
cultures and the importance of ensuring faculty commitment. 

o Finally, as the field has matured, interest has inevitably moved 
towards assessing assessment. What effect is this massive effort 
having on what students learn? On campuses that have engaged in 
assessment, faculty are more likely to think about teaching in terms 
of student learning. Some are lecturing less and asking students to 
write more. There is a new willingness to experiment with tech- 
nology for teaching and testing. Most importantly, one sees an 
emerging sense of faculty and institutional responsibility for 
student learning, and a commitment to monitoring and improving 
what students know and can do. In institutions where assessment 
is valued and experimented with, the conversation about teaching 
and learning has gained in intensity and sophistication, and 
ultimately students are bound to profit. As yet, though, the causal 
link between assessment and student learning has not been clearly 
established. 

The ever increasing requests for information and the overflowing 
audiences at national and international workshops testify to the con- 
viction in the higher education community that assessment, directly or 
indirectly, benefits what and how much students learn. The most ob- 
vious sign of assessment's grip on the academic consciousness is that 
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at present both AAHE's Assessment Forum and UTK's Assessment 
Resource Center are not only continuing but expanding their work. 

Project Continuation 

UTK's Assessment Resource Center has been institutionalized by 
the University as the Center for Assessment Research and Develop- 
ment. The Center has sponsored a survey administered by six 
Tennessee institutions to employers of their graduates, and a study of 
possible adaptations of W. Edwards Deming's quality improvement 
philosophy to higher education. FIPSE is now also funding an 
Assessment Clearinghouse at UTK to evaluate and disseminate 
assessment instruments and methods. 

The AAHE Assessment Forum has completed a second 3-year cycle 
which included, among other activities, the formation of an "Assess- 
ment Leadership Council." This group of practitioner-scholars 
examined research and development issues such as training materials, 
the relationship between assessment and accreditation and the role of 
assessment in pre-collegiate reform. A document entitled "Statement 
of Principles of Good Practice for Assessing Student Learning" was 
developed by this group. Printed copies of the Statement are available 
free (thanks to the Exxon Foundation) in packets of 25 from AAHE. 
AAHE has com- mitted itself to continuing the Assessment Forum 
with its own funds. 

The AAHE and UTK initiatives were prompted by an overwhelming 
need on the part of the higher education community at the inception 
of the assessment movement. Seven years later the clamor for 
guidance and information, far from abating, resounds abroad as well as 
throughout this country. Assessment has raised questions about 



teaching and learning that will stimulate and inform the international 
academic dialogue well into the 21st century. 



Available Information 

For further information about AAHE's Assessment Forum, write to: 

Karl Schilling, Director 
AAHE Assessment Forum 
One Dupont Circle, C ute 360 
Washington, DC 200c -111C 
202-293-6440 



Previous AAHE Forum Directors under FIPSE funds are: 
jazbarn D. Wright 

Department of Modern and Classical Languages 

University of Connecticut 

U-57 

Storrs, CT 06269 
203-486-1531 

Pat Hutchings, Director 

AAHE Teaching Initiative 

American Association of Higher Education 

One Dupont Circle, Suite 360 

Washington, DC 20036-1110 

202-293-6440 
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For further information about UTK's Assessment Resource 
Center, write to: 

Trudy W. Banta 

Vice Chancellor for Planning and Institutional 
Improvement 

Indiana University - Purdue University at Indianapolis 

140 Administration Building 

355 North Lansing Street 

Indianapolis, IN 46202 

317-274-4111 



AUSTIN PEAY STATE UNIVERSITY 

Project for Area Concentration Achievement Testing (PACAT): 

Curricular Evaluation 



Purpose 

In recent years many institutions have found that national 
achievement tests were poorly suited to curriculum improvement 
within academic departments. Typically, national testing instruments 
are not adapted to the goals and content emphases of particular 
disciplinary departments, and cannot gauge the impact of the cur- 
riculum on student learning. On the other hand, the validity of "in- 
house" tests, internally normed and tailored to individual departmental 
needs, cannot be judged against external criteria. 

The Project for Area Concentration Achievement Testing (PACAT) 
responded to this need for assessment in the major that is sensitive 
both to department characteristics and curriculum, and to the need to 
compare student performance across peer institutions. Begun in 1983 
as a consortium of Tennessee psychology departments, it aimed to 
expand and disseminate its assessment model in psychology, political 
science, and social work to include faculty in nine other states and 
create a minimum of six additional disciplinary consortia. 

Innovative Features 

PACAT surveys were used to identify content area emphases of 
departments in each discipline. Then assessment instruments for 
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graduating seniors were constructed to conform to these curricular 
patterns from items submitted by faculty at participating institutions. 

By giving faculty ownership of multiple-choice standardized testing 
and by measuring the relationship between content area performance 
and departmental curricula, PACAT bridges the gap between parochial 
instruments and nationally-normed exams. PACAT creates, maintains, 
and updates test items and serves as a coordinating office for scoring 
and interpreting the summary test results to departments. Score 
reports generated by PACAT contain raw and standardized scores for 
each content area tested and the individual department's performance 
history. 

These senior tests have been used to meet state-mandated 
assessment requirements. Since they point out the academic weak- 
nesses and strengths of graduating students, individual departments 
can alter course content and balance course requirements in line with 
test results. Further, departments can evaluate the impact of their 
curriculum on student achievement against other campus departments 
and against their own performance in previous years. 

Evaluation 

Several construct validation studies of the Area Concentration 
Achievement Test (ACAT) in psychology were conducted by independ- 
ent evaluators and by faculty at adopting institutions; that is, 
evaluators examined the extent to which inferences and decisions 
derived from test scores were supported by evidence and rationale. 
Comparison of groups of introductory and senior students showed 
seniors' test performance to be superior on all subtests and on overall 
scores. The ACAT was thus demonstrated to be psychometrically 
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sound and to measure what it was designed to measure~the impact of 
the psychology curriculum on student majors. Also, reliability 
statistics were collected at each test administration by using multiple 
versions of the test and separate groups of participating students. 

Evaluation of PACAT shows that the project exceeded its objectives 
in the following areas: number of curriculum surveys returned; 
number of follow-up requests; number of ACAT instruments 
administered annually; number of disciplines and curricular patterns 
used by ACAT; number of institutions and states added to the 
consortia; and test items added to item pools used to construct ACAT 
instruments. 

When FIPSE funding began in 1988, 865 ACATs had been 
administered in 22 departments during a five-year period, mostly in 
Tennessee. By 1992, 5,267 ACATs had been administered in 54 
departments in 19 states. Curriculum surveys were sent to 10,600 
academic departments nationally and over 4,700 were returned in 13 
disciplinary areas. Follow-up surveys were requested by 1,394 
departments. The Educational Testing Service contributed to PACAT 
a large pool of items gathered under another FIPSE-funded project. 

Beyond the three original ACAT disciplines, new instruments have 
been introduced in English literature, communications, art, biology and 
agriculture. Five new curricular versions of the ACAT in psychology 
and two in biology have been adopted for use. New multi-state 
consortia are being formed in history, public administration and 
criminal justice. 

Of course, the long-term impact of PACAT on students will depend 
on the extent to which test results are used to improve curriculum and 
instruction. It is too early to determine this effect definitely, since the 
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consortia are still forming and it takes several years of data collection 
before departments are ready to act on the results. 

ACAT has already been used to justify program accreditation, 
perform self-studies and comply with state-mandated assessments. 
Colleges and universities such as Jamestown College, Belmont College, 
The University of Alabama, Ohio State University, Wayne State 
University and East Tennessee State University have used ACAT 
results to support faculty development, to isolate areas of academic 
weakness among their graduating students, to readjust curriculum 
content to balance those weaknesses, to initiate faculty and student 
research, and to provide external validation to program evaluation of 
departments and courses. 

Project Continuation 

Austin Peay continues to support PACAT at levels somewhat higher 
than those prior to FIPSE, but it must now obtain independent 
funding for ACAT test materials and administration. Presently, 
PACAT is continuing its complete program and developing new 
assessment consortia. 

Major Insights and Lessons Learned 

Giving individual departments a voice in the construction of tests to 
evaluate their curricula is a long, slow process, taking at least three or 
four years to implement programs, many of which then require several 
years of fine tuning and coordinating efforts with administrators and 
departmental faculty. Unfortunately, many institutions ignore stu- 
dents while planning assessment, even though that is a time when 
they could gain their acceptance and cooperation. 
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PACAT revealed that many departments at small institutions are 
not able to initiate assessment programs of their own, especially those 
using external comparisons. By pooling resources and costs through 
consortia, however, assessment can be performed and the diversity of 
departmental curricula can be maintained. 

Available Information 

Information about the surveys, curricular patterns, and the ACATs 
in specific disciplines is available by writing or calling PACAT. 
Departments are welcome to join consortia at any time and, although 
submitting test items is not a requirement, they are encouraged to do 
so. Upon receipt of a written request, sample test booklets will be 
provided for examination. Commentaries, both favorable and 
unfavorable, combined with concrete examples for improvement of the 
test items, are welcome. 

Anthony J. Golden, Director 

Project for Area Concentration Achievement Testing (PACAT) 
P.O. Box 4568 

Austin Peay State University 

Clarksville, TN 37044 

615-648-7451 BITNET; ANTHONY@APSU 
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CITY UNIVERSITY OF NEW YORK 



The National Project on Computers and College Writing 
Purpose 

This evaluation and dissemination project aimed to assess the 
effectiveness of computer-assisted composition programs and to 
showcase outstanding college programs that employ this instructional 
strategy. A network of 15 competitively selected institutions of all 
sizes and types, chosen from among 90 applicants, participated by 
joining in a centrally designed assessment program and by sharing 
their computer-use models and instructional strategies with each other 
and with wider audiences. 

Project Activities 

Each of the 15 institutions agreed to carry out an assessment 
activity according to a common plan. Each site identified at least six 
sections of English composition, three taught using computer-assisted 
instruction and three by conventional methods. Each section enrolled 
similar kinds of students within a single institution, but this was not 
necessarily the case across institutions. A pre-course and post-course 
test common to all 15 sites was administered to each student, who in 
addition completed the Descriptive Test of Language Skills Sentence 
Structure Subtest and questionnaires regarding attitudes toward 
writing and writing anxiety. Each participating institution gathered 
site-specific qualitative data. The final sample consisted of 1,700 
students, equally distributed between computer-instructed and regular 
sections. 



The essays were evaluated centrally, all 1,700 holistically and 10% 
using analytic assessment methods. While overall the holistic scores 
showed no difference between experimental (computer instruction) and 
control (traditional instruction) groups, the scores did reveal 
significant advantages of computer instruction for those in 
developmental classes, adult learners and community college students. 
The analytic scoring did show significantly better performance by 
students in the experimental sections. Students in computer-based 
classes also showed a significant reduction in writing anxiety as 
compared with those in the control groups. 

The dissemination activities of the project were concentrated in a 
conference, "Computers and College Writing: Curriculum and 
Assessment for the 1990 > s," and a monograph, Computers and Colleg e 
Writing: Selected College Profiles . The conference enrolled 600 
participants from North America and Europe in June, 1990 and was 
followed by additional conferences in fall, 1990 and spring, 1992. The 
monograph contains accounts of computer-bas?d composition 
technology and instructional strategies at 49 institutions in the United 
States. 

Tnnovatiye Features 

The project is unique in its efforts to assess the effects of computer 
instruction on student learning and attitudes across a large number of 
institutions of varied size and type. No base of comparable size exists. 
The project both addressed questions about the value of computer- 
based composition instruction and created opportunities for the 
exchange of information about the way computers are being used in 
many institutions. Thus it provides an experimental base to justify 
the use of computers in teaching introductory composition courses as 
well as furnishing practical resources for teachers and institutions. 
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Evaluation 



The assessment plan and its implementation were not themselves 
the object of independent external evaluation. Project personnel have, 
however, been careful to point out the problems inherent in carrying 
out such a study and employing the kinds of assessment described. 
Specifically, they point to problems inherent in holistic assessment. 
While the methodology, when used with trained evaluators, works well 
in handling a large volume of essays, when used in a pre-test/post-test 
situation it tends to produce a regression (and advancement) toward 
the mean pattern. To counteract this effect the project used a number 
of measures of st udent growth in addition to the essays. 

Holistic scoring, in focusing on overall impressions, fails to identify 
particular advances in student writing, such as organization, copi- 
ousness, or stylistic sophistication. For this reason, 10% of the essays 
were subjected to analytic assessments, which revealed more signifi- 
cant differences between control and experimental groups. This result 
led project staff to assess the entire sample analytically, an effort still 
in process. 

Project personnel were well aware of the problems inherent in 
trying to measure gains over a single semester of instruction, but the 
logistical problems of maintaining contact with students and admin- 
ister a common assessment some semesters later were impossible to 
manage. 

Project Impact 

The number of participants in the conferences sponsored through 
the project and subsequent to its completion reflects its far-reaching 



effects. Project activities and results have also become known through 
numerous articles written and produced by project personnel. 

The project has produced a firmly established network of institu- 
tions engaged in computer-based instruction, and a data base of both 
institutional strategies and assessment results. This information is a 
valuable resource for colleges and universities to justify purchase of 
computers for composition instruction and for those seeking informa- 
tion on the technology, models and strategies of such instruction. 

Unanticipated Pr oblems 

Apart from the difficulties in finding fully satisfactory and 
manageable assessment mechanisms, which were acknowledged in 
advance, the project worked much as planned. Integrating the large 
volume of information gathered, an effort that continues to the 
present, required considerably more time and energy than expected. 

Major Insights and Lessons Learned 

The project has produced convincing evidence of the value of 
computer-based composition instruction, particularly for less well 
prepared groups of students. 

Project Continuation 

In the spring of 1992, the project sponsored "Computers Across the 
Curriculum: A Conference on Technology in the Freshman Year," 
which addressed computer-based instruction in a range of introductory 
college courses. 
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An effort is now underway tG use the project's findings as the basis 
for a program of technology-driven writing instruction in the New 
York City Public Schools. 



Available T nfnrmfli ™ n 



The project has generated a number of articles, as well as the major 
monograph, Computer and College Writing: Selected College Profiles . 
General information about the project and copies of the publications 
may be secured by writing to: 

Max Kirsh, Director 

National Project on Computers and College Writing 

Office of Academic Computing 

City University of New York 

555 W. 57th Street, 14th Floor 

New York, NY 10019 

212-541-0320 



Each of the project sites has developed curricular materials, from 
videotapes to course syllabi. These are available from the individual 
institutions or, in some cases, from the project director, who will 
gladly supply the list of participants. 
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HARVARD UNIVERSITY 

The Harvard Assessment Seminars: Exploration with Students and 
Faculty about Teaching, Learning, and Student Life 

Purpose 

The Harvard Assessment Seminars constitute the University's 
response to President Bok's 1986 call for program assessment designed 
to lead to policy recommendations. The ongoing Seminars gather 90 
faculty and administrators from 15 northeastern colleges, and 60 
Harvard graduate and undergraduate students. 

Transcending the mere investigation of student knowledge, the 
Seminars aim to foster curricular and pedagogical experimentation and 
innovation. Participants congregate in small working groups that 
include at least one faculty member, one administrator and one 
student. Each group identifies a project, carries it out and evaluates 
it. 

The groups that met during the 1986-1989 FIPSE grant period 
examined questions ranging from what undergraduates think makes 
for a good course or a helpful adviser to how involvement in athletics 
or employment affects academic performance. Various samples of the 
undergraduate population were interviewed, and groups of freshmen 
and upperclassmen were asked to keep time logs of their activities. 
Harvard and Radcliffe graduates now in their 30's, 40's and 50's were 
surveyed regarding their opinions of their college experience. 
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Project IfflBSgfc 



The Seminars' principal achievement has been the creation of an 
atmosphere of experimentation and innovation in teaching and the 
curriculum. As a consequence of the Seminars' findings and 
discussions, faculty are dividing students into small study groups, 
asking them to write one-minute papers at the end of class, and giving 
more quizzes and returning them more promptly. 

As the project continues, its results will be translated into policy 
recommendations. In the meantime, assessment has become part of 
the Harvard campus culture, and the results of the investigations, 
although they frequently confirm what had already been established by 
research elsewhere, have been embraced by the faculty because they 
reflect the immediate campus reality. 

Major Insights and Lessons Learned 

The surveys and interviews yielded a collection of insights, some 
large, some seemingly small, but all of them pertinent to the process of 
teaching and learning. At the prompting of student participants, for 
example, one group undertook a study of gender differences in the 
perception of college. Results showed a much greater divergence 
between the sexes than had been anticipated: male and female 
students differ in their self-image, in the qualities they seek in their 
advisors, and in the way their academic performance affects their 
satisfaction with college. They also study differently: although both 
men and women (and especially female science students) perform 
better if they work in groups, women are much more likely than men 
to study alone. 
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Students show a high degree of consensus on what makes for a 
good course: frequent, detailed, punctual feedback that allows for 
revision. Although involvement in co-curricular activities does not 
affect academic performance, there is a strong correlation between 
such involvement and satisfaction with college. The passing decades 
have little effect on graduates' perception of their college experience: 
Harvard and Radcliffe alumni in their middle years express much the 
same views as current undergraduates. 

One group's research debunked the venerable myth of faculty 
inaccessibility. Junior instructors and senior professors alike showed 
an almost wistful eagerness to meet with students who take even the 
most modest initiative, and they also turned out to have a lively 
interest in teaching innovations. Faculty did express reluctance, 
however, to adopt reforms that would necessitate adding material to 
already overloaded courses and to incorporate technology without the 
benefit of proper training. 

What can the Seminars teach the higher education community 
about how best to conduct assessment? At Harvard as on campuses 
across the nation, faculty involvement-- n ownership n of assessment by 
the men and women who write the syllabi and give the lectures-was 
the primary factor in success of the Seminars. 

Despite the predominance of assessment as a topic of discussion 
during the last decade, not every academic has a clear idea of its 
meaning. Especially for faculty at private institutions, assessment 
carries connotations of standardized all-purpose instruments at best, 
and state-mandated "gateway" examinations at worst. Thus, it is 
important that every college define precisely what assessment should 
mean to its teachers and learners. Harvard faculty chose to extend 
the concept of assessment beyond simply finding out what students 



know, and embraced a philosophy that encourages innovation and 
creativity both inside and outside the classroom. 

Student involvement in assessment can be extremely valuable. 
Although students initially were not invited to take part in the 
Seminars, many graduates and undergraduates soon began to manifest 
a desire to assist in the work. Their collaboration proved providential: 
they did much of the basic research and statistical work, interviewed 
their peers, and contributed energy and insights. In exchange they 
received a modest stipend and an unparalleled opportunity to work in 
a quasi-collegial capacity with faculty. 

The Seminars began in response to presidential interest in assess- 
ment, and they would have been unable to continue in the absence of 
intellectual and financial support from the highest administrative 
levels. In addition, the Seminars gained impetus from the clear 
understanding that the various discussions and explorations would 
result in policy changes that would affect the very core of the 
educational process at Harvard. This assumption provided the conti- 
nuing focus for the activities of each group. The mixed composition of 
the groups-faculty, students and administrators-ensured that the 
viability of each project was considered hand in hand with its academic 
merits. 

A frequent outcome of the institutionalization of assessment is the 
creation of a propitious climate for ongoing experimentation, with its 
attendant disappointments as well as triumphs. This openness to fail- 
ure as well as success, this acceptance of the inevitable chaotic aspects 
of intellectual ferment is basic to the proper cultivation of creativity. 
As the Seminars continue the work of assessment on campus, no dead- 
line is envisioned for their task, for it has become a major thread in 
the fabric of teaching and learning at Harvard. 
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Project Continuation 



Nourished by the campus climate and spurred by the results of 
completed projects, the members of the Assessment Seminars continue 
to engage in various undertakings, many of them designed to put into 
practice the strategies suggested by the research projects outlined 
above. Thus, some groups are exploring how to improve advising, 
especially in view of gender differences. There is interest in promoting 
faculty-student interaction outside the classroom, and in finding out 
how faculty can encourage students to take academic risks. Still other 
participants are investigating how to put alumni wisdom to use in 
guiding undergraduates' academic choices, and how to internationalize 
the student body and the curriculum, with particular emphasis on the 
full integration of foreign students. 

Recognition 

The Harvard Assessment Seminars have been featured in The New 
York Times as well as a number of academic publications such as the 
AAHE Bulletin . Over 3,000 copies of the 1990 Report, written by the 
leader of the Seminars, Richard J. Light, have been distributed. The 
second Report, published in 1992, has also had wide circulation. 

Available Information 

The Harvard Assessment Seminars' Reports are available from: 

Richard J. Light 

Harvard Graduate School of Education 
Larsen Hall 

Cambridge, Massachusetts 02138 
617-4954183 
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HARVARD MEDICAL SCHOOL 



A Randomized Controlled Trial Evaluating the Impact 
of the New Pathway Curriculum 

Purpose 

Despite many calls over the last fifty years for reform of the 
unintegrated, lecture-dominated curricula characteristic of American 
medical schools, few schools have embarked on change. Researchers in 
the Office of Educational Development at Harvard Medical School 
believed that the reluctance to reform the curriculum was in part 
attributable to the absence of any assessment of non-traditional forms 
of medical education that included a true control group* The purpose 
of this project was to carry out just such an assessment. 

Project Activities 

In 1985, Harvard initiated the New Pathway curriculum. This 
curriculum emphasized active, problem-based, student-directed learning 
in small groups. The program's learner-centered instructional 
strategies stood in contrast to the passive, large-group lecture method 
in use at most medical schools. In the New Pathway, basic science and 
clinical medicine were integrated throughout the four years, with 
students interacting with patients from the first week, and returning 
to basic science even in the fourth year. Each New Pathway partici- 
pant developed a multi-year relationship with a generalist preceptor. 
The program was based on the notion that learning is facilitated when 
knowledge, attitudes and skills are taught together in relation to real 
clinical problems. 



Students in the New Pathway Program were randomly selected 
from a list of volunteers. Those not chosen went through the 
traditional program with the non-volunteers. Thus three groups were 
available for comparison: those selected for the New Pathway Pro- 
gram, those who volunteered for the program and were not selected, 
and those who opted for the traditional program. 

The project involved a comparison in six major areas between the 
volunteers who were selected for the New Pathway Program and those 
who volunteered but were not selected. The six areas were: 1) entry 
characteristics and study participation; 2) the cognitive domain: factual 
knowledge and learning behaviors; 3) clinical skills, including 
reasoning and problem solving; 4) psychosocial orientation; 5) stu- 
dents' personal experience of the old and new curricula; and 6) career 
choice. Students entering in 1985 and 1986 were followed for four 
years to assess both their responses to their educational experiences 
and the differences in outcomes for the two groups that might be 
associated with the differences in the design of their education. 

Innovati ve Features 

The students who volunteered for the New Pathway Program but 
were not selected provided a true control group. The two groups were 
of equal size and well matched on a wide range of factors including 
demographic characteristics, college grade-point average, research 
experience, Medical College Admission Test scores and several 
psychosocial characteristics. 

More importantly, the control group could be presumed to have 
applied to medical school with the same orientation toward medical 
education as the experimental group: they had, after all, volunteered 
for the new program and had been assigned to the traditional one as 
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the result of a random selection process. Interestingly, though, by the 
time they arrived on campus the non-selected volunteers (control 
groups) showed a lower level of preference for discovery-style learning 
and a more positive orientation toward teacher-directed learning. This 
statistically significant difference may have resulted from chance or 
could have reflected an early adaptation to the methods and expecta- 
tions of the curriculum to which these students had been assigned. 

The researchers used a wide variety of bases for comparison of the 
two groups (34 in all) including demographic data, standardized test 
data, psychological measurement instruments, standardized patient 
assessments, cognition and reasoning tests, student statements, 
interviews with students and student participation in various sorts of 
learning activities. They were thorough and got surprisingly good 
cooperation from their subjects. 

Evaluation 

Though no formal evaluation of the assessment activity which 
constituted the project itself was undertaken, the project's complete- 
ness, care of design, and careful interpretation of findings give it 
substantial surface credibility. 

Project Impact 

The project's confirmation that the New Pathway Program was 
producing medical school graduates with levels of knowledge and 
clinical skills comparable to those in a traditional program at this 
highly selective medical school is an important finding. Such a 
conclusion can serve to reassure skeptical faculty that, at least in the 
case of the very brightest, a curriculum that is more student-centered, 



fosters more active learning, provides better integration of basic 
science with clinical practice and places more emphasis on the social 
and psychological aspects of medical practice produces at least as high 
a degree of technical competence as the more traditional program. 
The demonstration that students develop significantly higher levels of 
humanism, as measured by a broad array of assessments at the end of 
year two, should affirm that enhancement of psychosocial competence 
can be achieved without measurable loss of technical biomedical 
competencies. These findings should help answer the question "Can 
we teach humanism?" with a strong "yes," in years one and two. 

If the program's other outcomes (see below) are seen as advantages, 
this research can reinforce the efforts of medical curriculum reformers. 
It is too soon to know whether this work will contribute to such an 
agenda or not. One indication may be that, at least at Harvard, the 
entire curriculum has shifted to a more student-directed, problem- 
based format since the program has been in effect. Faculty who have 
had experience with the program tend to adopt its strategies. 

Unanticipated Pfnhj epia 

While students participated at high levels during their first two 
years, data from the last two were more sparse. Students did not 
complete some of the instruments that required time and thought, or 
participate in certain exercises, such as standardized patient inter- 
action or clinical problem-solving. Participation of larger numbers 
might have demonstrated more marked differences between the two 
groups, providing stronger affirmation of tendencies that are based 
more on convergence of findings than on sharp differences in data. 
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Major Insights and Le ssons Learned 



In addition to demonstrating an equivalence in knowledge and 
clinical competence between students in the traditional and non- 
traditional programs, the program evaluation revealed that students in 
the non-traditional program learned and employed more psychosocial 
skills, including more advanced skills in relating to patients during the 
first two years of study, and had more positive attitudes about the 
psychosocial aspects of their relationships with patients. Whether 
these effects persist to the end of the clinical study years could not be 
established due to the reduced size of the sample. New Pathway stu- 
dents preferred the student-directed learning environment and 
developed an ability to study for understanding rather than engaging 
primarily in rote memorization. The comparative lack of structure in 
the new program was more stressful, but students were more chal- 
lenged and less bored by it. They also knew and were better known 
by their faculty during the preclinical years. 

Recognition 

The New Pathway Curriculum was the subject of a NOVA program 
on the Public Broadcasting System. Since completion of the project, 
faculty who participated in this study of the New Pathway received a 
grant from the Robert Wood Johnson Foundation to evaluate their 
new national initiative "Preparing Physicians for the Future," 

Project Continuation 

The Office of Educational Development at Harvard Medical School 
has developed an internal program evaluation that is supported by its 
operating funds and additional extramural support. 



33 



Avajfchlp Inform ation 



Two research reports, several smaller papers, the FIPSE final 
report, and course materials are available from: 

Gordon T. Moore 

Office for Educational Development 
Medical Education Center 
Harvard Medical School 
260 Longwood Avenue 
Boston, MA 02115 
617-421-2747 
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MIAMI UNIVERSITY 



Assessing Models of Liberal Education: 
An Empirical Comparison 

Purpose 

Postsecondary institutions today are well aware of the importance 
of curricular assessment. When, as in the case of Miami University of 
Ohio, two alternative approaches to general education coexist on the 
same campus, the task of evaluating the outcomes of the two curricula 
becomes even more pressing. 

Miami University wanted to distinguish the effects on student 
intellectual and personal development of two general education 
curricula: the University Requirement, a disciplinary, distribution- 
based general education program (replaced in fall 1992 with a new 
liberal education program), and the Western College Program, an 
interdisciplinary core curriculum. 

Innovative Features 

Miami administered a dozen different tests to matched groups of 
first, second and fourth year students enrolled in the disciplinary and 
the interdisciplinary curricula. Qualitative and quantitative measures 
were used, and cross-sectional as well as longitudinal data were 
collected and analyzed. As a result of this many-sided approach, a 
complex picture of the students emerged, along with certain patterns 
of strengths and weaknesses. 
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Evaluation 



In general, measures of general liberal arts skills-the ACT-Comp 
(ARC version), the Academic Profile, the Test of Thematic Analysis, 
the Analysis of Argument, the Measure of Epistemological Reflection 
and the SUNY-Fredonia Tests of General Education-showed few 
significant differences between students engaged in the interdisci- 
plinary curriculum and those in the disciplinary curriculum. 

While performance on most measures was quite high, the results 
did suggest a need for students in both groups to increase their 
understanding of the scientific method and improve their knowledge of 
global issues. It also became apparent, from the Measure of 
Epistemological Reflection, that interdisciplinary students enter and 
graduate at a higher cognitive level than their disciplinary peers. The 
test's author speculated that Miami faculty in both programs may 
orient their teaching to a higher cognitive level than most students 
have achieved. 

Several of the instruments used seem to have been designed for 
students of lower academic ability than Miami's, with the consequence 
that first year students received extremely high scores, leaving little 
room for sophomores and seniors to show gains. Several faculty 
members argue that many of the instruments used measure only 
traditional linear reasoning, and that in some cases higher-level 
reasoning processes exhibited by advanced students may have been 
evaluated negatively. 

The most significant differences between the two groups of 
students emerged from tests of cognitive development and of specific 
academic behaviors and values-the Myers-Briggs Type Indicator, the 
ACT Activity Inventory, the College and University Environment 
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Scales, the College Characteristics Index, the College Student 
Experiences Questionnaire, and the Cooperative Institutional Research 
Profile. Data show that whereas students in the disciplinary 
curriculum exhibit higher levels of involvement in social and athletic 
activities, Western College students participate more frequently in the 
intellectual, artistic, political and human service activities of the 
campus. Although this category of instruments did reveal significant 
differences between the two groups, many of these differences were 
foreshadowed by entry data from the Cooperative Institutional 
Research Profile (CIRP) before the students enrolled. 

Project Impact 

As a result of the project, awareness of the importance of 
assessment and of the national attention and financial rewards that 
assessment programs can bring to institutions grew considerably on 
campus. New faculty were hired to strengthen instruction in scientific 
and quantitative reasoning. "Expectation Statements," written each 
semester by the interdisciplinary students only, yielded much informa- 
tion on how students experience their lives on campus, and led to 
some changes in the treatment of the sophomore year, particularly the 
development of the students' upper-level program of study. Further, 
because the tests were shown to have serious shortcomings for Miami 
students, faculty were stimulated to continue seeking better ways to 
measure the effects of general education (see below). 

Unanticipated Problems 

Because of the intensive and prolonged nature of the testing, it was 
difficult to ensure that students continued to participate. Thus, be- 
cause of considerable attrition, the groups that began as highly 
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matched ended with a much lesser degree of matching. The task of 
transcribing the interview tapes also turned out to be inordinately 
laborious, and delayed the availability of an important source of data. 

M ajor Ing ighta an d Lessons Learned 

Although the project did not demonstrate consistent, significant 
academic differences between the two groups of students, clear 
differences did emerge in the areas of campus values, academic 
behaviors, involvement in learning and interpersonal interactions. It 
is not possible to tell, however, whether these differences result from 
the respective curricula, or from self-selection on the part of the 
students. A "chicken or the egg 11 conclusion was reached to the effect 
that campus cultures either shape the academic experience of students 
or are shaped by them. 

Perhaps the most significant insight of the project for Miami 
University was related to the inadequacy of nationally available 
standardized liberal arts skills tests for measuring the different effects 
on student learning of the disciplinary and interdisciplinary curricula. 
This may be due to the high caliber of the populations being 
measured, which makes it very difficult to obtain gain scores large 
enough to differentiate between the two groups. On the other hand, 
the tests may not be sufficiently refined to measure the relatively 
subtle differences between the outcomes of the two curricula. It is 
possible that the curricula result in differences not in liberal arts 
skills, but in the behaviors and values mentioned above. It is also 
possible that the specific nature of the curriculum has less influence on 
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student learning than do campus ethos, student self-selection, class 
size, and other non-curricular factors. 

The national standardized tests do, of course, provide national 
norms for student performance. Because they show a high degree of 
correlation with college-entrance scores, they may constitute more 
meaningful measures of recruitment efforts than of curricular effects. 

Project Continuation 

Two parallel portfolio programs implemented in fall 1992 assess 
both the Western College Program and the new liberal education 
curriculum. All students in the Western Program are required to keep 
portfolios. A random sample of 40 students campus-wide has been 
recruited out of each entering class since the summer of 1990 to 
participate in the project. Results have been shared via the liberal 
education newsletter, "The Miami Plan." A campus-wide Assessment 
Council developed a Statement of Philosophy and Goals for Assess- 
ment at Miami University, and the University Senate passed it 
unanimously in 1992. Implementation of assessment within the 
academic departments was the focus of the Assessment Council's work 
during the 1992-93 academic year, supported by a State of Ohio 
Program Excellence Grant and the Miami Liberal Education Program. 

Available Information 

In addition to its 1990 final report to FIPSE, the project has 
resulted in over 20 conference and workshop presentations. Several 
reports of project results, plus materials on the continuation of the 
project since FIPSE funding expired, are available: 1) Excerpts from 
the liberal education newsletter, "The Miami Plan,' 1 on portfolios, 
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student time use, interviews, and free-writing exercises; 2) the Miami 
Statement on Philosophy and Goals for Assessment; 3) the 1992 
Summer Orientation talk to parents sharing assessment results; and, 
4) the October 1992 AAHE Bulletin article, "AAHE's Assessment 
Forum Changes Hands," pp. 10-14, that provides observations about 
the Miami project and subsequent work. A chapter written with 
SUNY-Fredonia on the two FIPSE projects will appear in the Jossey- 
Bass book, Are We Making a Difference? edited by Trudy Banta. 

For further information please contact: 

Karl Schilling, Director 
AAHE Assessment Forum 
One Dupont Circle, Suite 360 
Washington, D.C. 20036-1110 
202-293-6440 
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THE STATE UNIVERSITY OF NEW YORK COLLEGE 

AT FREDONIA 



Improving Undergraduate Education: 
The Development of College-Wide Measures of Progress 
Toward Goals of General Education 



Purpose 

While curricular innovation is most frequently beneficial, if only 
because of the discussion and reflection it generates among faculty, 
rigorous assessment is essential in order to ascertain whether the new 
course of study has brought about the intended improvements in 
student learning. When SUNY College at Fredonia instituted a skills- 
oriented general education program (the General College Program, or 
GCP) in the early 1980s, the College made a commitment to assess the 
curriculum once it had become well established. The project was 
designed to investigate whether students were learning what faculty 
thought they were teaching, and whether the learning that was 
occurring resulted from the instruction the students received. 

Since the GCP does not require all students to take the same 
courses, ways had to be found to measure skill acquisition that would 
not depend on academic content. Moreover, Fredonia faculty and 
administrators were determined to ascertain whether gains in student 
growth could be attributed specifically to their teaching rather than to 
the normal process of maturation. Before work could begin on any of 
tt ese issues, however, faculty had to define the skills and under- 
standings that students were expected to develop in sufficient detail to 
allow measurement. 



Innovative Features 



Persuaded that the assessment of liberal education should be an 
exercise in it, the proposal to FIPSE granted the twelve faculty 
members of the Assessment Committee ample time to engage in 
extensive study and reflection on the topic of assessment. At the 
outset, none of the committee members was an expert in the field. 
Nevertheless, as liberally-educated individuals and the ultimate experts 
on their own campus and students, they confidently embarked on the 
exploration of assessment. As a result, the project generated a sense 
of intellectual discovery-and therefore commitment--among faculty 
that would have been impossible without the opportunity to undertake 
intensive study and discussion. 

The group's first decision was to reject standardized tests. 
Designed for students from infinitely varied backgrounds and diverse 
types of institutions, the instruments did not adequately match 
Fredonia's curriculum, and faculty believed that the objective format 
would test recognition rather than original thought or the ability to 
manipulate ideas. 

The Committee members concluded that they would develop their 
own short-essay tests of the skills addressed by the curriculum: 
reading, writing, reflexive thinking, scientific reasoning, mathematical 
problem-solving, and socioethical understanding (critical thinking was 
subsumed in the preceding categories). The tests would be admin- 
istered to freshmen and upper classmen as well as to a comparison 
sample composed of freshmen and upperclassmen at Miami University 
and Miami's Western College, selected because their high school 
achievement levels were thought to be similar to those of Fredonia 
students, and because at that time Miami University did not 
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have a skills-oriented general education program, but rather simply a 
distribution requirement. 

Knowing students' proclivity to provide the answers they believe to 
be expected of them, Fredonia's Assessment Committee strove to elicit 
genuine responses rather than professions of faith. Thus the tests 
used open-ended questions and posed dramatic situations (a European 
exchange student criticizing certain aspects of American life, for 
example) to which students responded spontaneously and often 
passionately. The blind scoring system also examined essays for 
qualities not explicitly mentioned in the questions-such as ethno- 
centrism, or an exclusive focus on the present. 

Recognition 

The careful and unusual design of Fredonia's assessment project 
quickly attracted notice. Within the SUNY system, Fredonia has 
become a leader in assessment, participating in four regional SUNY 
assessment workshops. Members of Fredonia's Assessment Committee 
are in frequent demand as consultants; one faculty member served in 
the Exxon-funded 1989-90 portfolio project, described in the American 
Association of Higher Education's Time Will Tell: Portfolio-Assisted 
Assessment of General Education . The College maintains a mailing 
list of requests for information about the project, has made pre- 
sentations at three AAHE assessment conferences and at regional 
conferences, and has conducted workshops for the South Carolina and 
Washington State assessment networks. A description of the project 
appeared in Assessment Update . Professor Karl Schilling of Miami 
University, currently director of the AAHE Assessment Forum, who 
collaborated with Fredonia and directed his own FIPSE assessment 
grant (see page 31), has integrated information about the Fredonia 
project in his own presentations and consulting. 



Prnj ftdh Impart 



The Assessment Committee produced nine tests, with scoring 
manuals, of reading, writing, reflexive thinking, scientific reasoning, 
mathematical problem-solving and socioethical understanding. These 
are instruments in which the faculty has confidence, that are 
sufficiently precise to allow for focused planning, and that can be used 
in the future to yield a longitudinal perspective on student progress. 
In the process of designing and administering these tests, faculty 
reflected on the purposes of the curriculum, learned about results of 
their program, and drafted recommendations for its improvement. 

The findings indicate that Fredonia students make least progress in 
critical thinking-particularly with respect to discerning their own or 
others* assumptions and biases— and in problem-solving. Students 
showed little awareness of themselves as learners, and exhibited poor 
scientific reasoning. They also remained quite ethnocentric in their 
views, and failed to establish connections among different courses as 
well as disciplines. 

Although the Miami University and Western College populations 
turned out to be less comparable to Fredonia students than had 
initially been hoped, the results of comparisons between the two 
groups showed a consistent pattern. In general Miami freshmen 
scored at the level of Fredonia upperclassmen. However, there was a 
greater difference between the scores of Fredonia freshmen and 
upperclassmen than between beginning and advanced groups at Miami 
University. 

This was cautiously interpreted at Fredonia as meaning that the 
curriculum, and the way it is taught, is responsible for the gains made 
by Fredonia students. This finding is all the more positive in light of 
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research that indicates that better prepared students (in this case, 
Miami University freshmen) grow faster academically than their less- 
prepared peers. 

Rather than suggest specific means of improvement, the recom- 
mendations on strengthening general education outlined by the 
Assessment Committee point to areas in need of attention. Two 
faculty study groups were formed to respond to these recom- 
mendations (see below). 

M ajor Insights and Lessons Learned 

The decision to dispense with outside experts and standardized 
tests not only produced custom-made instruments eminently suited to 
the campus and the curriculum, but resulted in a high sense of 
ownership of the project on the part of the faculty. This was 
enhanced by the visible commitment to assessment by administrators 
at the highest levels, and by the choice of highly respected faculty to 
serve on the Assessment Committee. 

The project also yielded much incidental information on student 
learning, which, while often puzzling, nevertheless gave faculty a 
deeper insight into teaching and learning at Fredonia. Finally, the 
task of assessment itself was instructive, forcing faculty to give shape 
to their ideas of what constitutes effective teaching, and giving 
students the opportunity to reflect and write on a number of issues, 
and thereby increase their awareness of themselves as intellectual 
beings. 

Participants in the project, having found standardized tests 
insufficient to their needs, welcome the national trend towards 
embedded assessment. Before this can succeed on a large scale, 
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however, scoring techniques for long-term qualitative measures of 
portfolios, essays, and recorded interviews must be developed. The 
project findings, and more than likely the collaboration with Miami 
University as well, have caused project participants to reflect on the 
need to develop ways of measuring the correlation between academic 
and extra-curricular involvement. 

Unanticipated Problems 

It was learned belatedly that, while both Fredonia and Miami 
University freshmen had high school averages of B+, the numerical 
equivalent of that grade was 86 in New York and 91 in Ohio. Thus, 
the Miami University students were better prepared academically than 
their Fredonia peers, and this affected the assessment process. 

In addition, although the purpose of the project was to find out the 
effects of the GCP specifically, it became impossible to distinguish 
whether the gams made by upperclassmen were due to the GCP 
courses alone or to other courses in the curriculum, especially since 
the College's entire program aims to develop the skills being 
measured. 

Project Continuation 

Two faculty study groups were formed to address the findings of 
limited student improvement in identifying biases and assumptions in 
reading and in their own thinking, and insufficient improvement in 
problem solving. As a result, workshops on problem solving have been 
instituted for faculty teaching introductory courses in mathematics. 

Some changes have been made in the GCP itself. The departments 
of Chemistry, English, Foreign Languages, History, and Political 
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Science experimented with portfolios as a way to help students 
improve reflexive thinking and make connections among courses. 
English and Foreign Languages have since made portfolios a 
permanent part of their programs. In general, the revised GCP 
emphasizes the attributes of thinking evaluated by the project, and 
increases the amount of attention to cross-cultural or international 
matters required in all fields. 

To assess the effect of these changes, the College is repeating its 
study. All nine tests were given to incoming freshmen in summer of 
1991, and will be repeated with a sample of these same students in 
spring of 1994. 



Available Information 



A substantial final report was filed with FIPSE at the end of the 
project. Two campus reports have been produced. The GCP and 
Student Learning: A Report To The Campus , appeared at the 
conclusion of the grant. The second, entitled GCP Study Group 
Reports , presents the conclusions of two faculty groups that met to 
examine the findings and recommendations of the Assessment 
Committee in the areas of critical thinking and problem-solving. For 
information about these reports, contact: 



Minda Rae Amiran 

Department of English 

State University of New York 

College at Fredonia 

277 Fenton Hall 

New York, New York 14063 

716-673-3125 
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THE STATE UNIVERSITY OF NEW YORK COLLEGE AT 

PLATTSBURGH 



Comprehensive Assessment in Academic Disciplines 

Purpose 

Assessment within academic disciplines must meet two seemingly 
contradictory criteria: on the one hand, it must reflect the particular 
perspectives of the departments involved; on the other, it must yield 
comparable data on student performance. 

To reconcile these divergent requirements, the State University of 
New York College at Plattsburgh organized an assessment consortium 
composed of five SUNY colleges: Buffalo, Cortland, Fredonia, 
Plattsburgh, and Potsdam. The colleges sent teams of two faculty 
from several, but not necessarily all, of the following disciplines: 
Biology, Teacher Education, Foreign Language and Literature, History, 
and Psychology. 

Trmnvative Features 

The purpose of the consortium was to develop comprehensive 
examinations in each of the five disciplines. To achieve this purpose, 
teams met over a period of three years and engaged in a step-by-step 
process that yielded considerably more than a set of assessment 
instruments. 

The teams began by defining the nature of their disciplines with 
respect to undergraduate learning, discussing not only content but the 
field's epistemology as well as its contributions to society, its purpose, 
values, and ethical standards. Faculty next examined the assessment 
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strategies most appropriate to each discipline, frequently discarding 
approaches grounded in traditional psychometric theory in favor of 
measures that seemed most accurately to reflect learning in each Held. 
Most importantly, at the conclusion of the process they engaged in 
discussions of the implications of assessment for curricular reform. 

The Biology group developed an assessment of the following goals: 
1) to provide students with broad-based general knowledge and 
familiarize them with the language of science; 2) to provide students 
with some basic laboratory skills and training in the proper use of 
standard instrumentation; and 3) to increase student awareness of 
current issues in science, especially the relationships of science to 
society and the ethical questions inherent in the use of technology. 

The Psychology group developed an objective multiple-choice test 
that meets rigorous standards of measurement. The instrument con- 
sists of three parts: 1) a content section that assesses knowledge of 
concepts, terms and principles in ten basic areas of Psychology; 2) a 
critical thinking or "process" section that assesses ability to analyze 
research; and 3) a section that assesses the relationship of course 
experiences to the scores on the first two sections. 

The Teacher Education assessment model included a 40-cell matrix 
generated by four domains of assessment and ten program elements. 
The Education group sought assessment strategies and data sources to 
label judgments in each of the cells of their matrix. They explored 
potential contributions from sources such as vignettes, portfolios, 
observations, video tapes, questionnaires, interviews, written exams, 
jury by peers, etc. 
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The History group developed and refined an assessment instrument 
which allowed students to reflect in various ways on what they have 
learned during an undergraduate course in history. Students are 
assessed on their synthesizing/analytical capabilities as supported by 
comparative historical reasoning and evidence. An interpretive essay 
assignment allows students to synthesize diverse historical information 
as they attempt to validate or challenge provocative generalizations. 
Students are asked in addition to demonstrate their command of 
factual contextual historical knowledge. A battery of eight multiple- 
choice tests allows them to choose areas of competency which they are 
certain to have encountered. The final version of the assessment 
includes questions on the nature of the historical discipline and its 
practices. 

The Foreign Language and Literature group focused its activities on 
six key areas embodied in all foreign language programs: oral pro- 
ficiency, writing skills, listening comprehension, reading proficiency in 
the foreign language, "proficiency" in literature, and knowledge of 
civilization and culture. The group constructed grids for each of these 
six areas modeled on the ACTFL/ETS guidelines for proficiency. Oral 
skills are assessed in an interview format, and the writing and literary 
interpretive skills through written assignments. A multiple-choice 
format assesses basic knowledge. 

Evaluation 

The project was evaluated on the basis of three goals. The first 
was the creation and maintenance of an inter-institutional consortium 
within a large public university system. Achievement of this outcome 
was demonstrated by the actual sustained efforts of the consortium. 
Furthermore, participants engaged in an analysis that generated in- 
sights into the dynamics of operating such a consortium. 



The second goal was the development of assessment-related con- 
ceptual frameworks for each discipline so that assessment strategies 
could be devised in accordance with a comprehensive understanding of 
the key elements and boundaries of each discipline. These frameworks 
were evaluated according to the nature of the discipline in question. 
In some cases, external reviewers were asked to comment on the 
quality of the work. In other cases, the participants made their own 
judgments based on comparisons with existing frameworks within their 
disciplines. 

The third goal was the development of assessment strategies for 
each discipline. The effectiveness of these strategies and instruments 
was again evaluated in different ways for each of the participating 
disciplines, but each involved some form of field-testing to verify its 
utility and viability. 

Project Impact 

The project resulted in the creation and maintenance of a con- 
sortium of institutions within a state university system. Not only did 
it yield many insights into the operation of such a system, but the 
coming together of faculty from different disciplines and institutions 
for the sake of a common endeavor produced in the participants an 
unanticipated sense of renewal and stimulation. Each disciplinary 
team developed an assessment-related conceptual framework that not 
only produced an assessment instrument as a direct result of the team 
activities, but one which can be used in the future to develop other 
measures. 
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Major Insights and Lessons Learned 



When faculty joined the consortium, they believed that they were 
about to engage in a team effort to construct comprehensive examina- 
tions based on the knowledge they already possessed of teaching and 
learning in their discipline. Instead, they found themselves participa- 
ting in an arduous intellectual exercise of questioning and rethinking 
not only matters related to their discipline, but the nature of teaching 
itself. As a result, participants became passionately committed to the 
work of the teams, and faculty development turned out to be one of 
the project's primary benefits. 

Those involved in organizing the consortium concluded that the 
following strategies are essential to the success of projects such as this 
one: 

1) Faculty must be given as much autonomy and freedom from 
bureaucratic interference as possible, 

2) Participants must be selected according to their commitment to 
the project; their knowledge of the discipline; and their ability 
to communicate, to work within a team, and to provide leader- 
ship in their own departments, 

3) Faculty must be given control over the data that the project will 
yield. This allays fears that assessment results might be used to 
harm individuals or departments, and allows the work to pro- 
ceed within the norms of scholarly inquiry. 

4) A retreat-like setting stimulates informal discussion and 
interaction, 

5) Occasional interdisciplinary meetings should be held, since 
contact among individuals from different disciplines fosters 
creativity. 
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6) An adequate stipend (in this case, roughly the equivalent of 
remuneration for one summer course) not only allows adminis- 
trative control over minimum compliance but, most importantly, 
conveys a sense that the work is valued. 

7) The performance of each team is largely dependent on its 
coordinator; therefore, these individuals need extensive training 
and support. 

8) The leadership and commitment of central administration are 
essential to the success of projects such as this one. 

Unanticip ated PrnhlPTpa 

As many who have engaged in assessment know, it is difficult to 
involve students in the process unless it is mandatory and minimum 
performance levels are specified. Unfortunately, not only does such an 
approach restrict the kinds and extent of skills and values examined, 
but it also may proscribe creative or experimental approaches. 

The traditional, more limited assessment measures usually yield 
information that is only of marginal value for improving teaching and 
learning. Yet assessment instruments should not be used to judge 
students or curricula unless their validity and reliability have been 
amply demonstrated, something that is of course more difficult to 
establish with locally-developed measures. The project yielded no 
solution to this dilemma, although participants emerged with a 
conviction that assessment is of great value in teaching and learning. 

When conducted in the ways outlined above, assessment is not only 
labor-intensive but expensive as well. Thus, despite its unquestionable 
success in fomenting discussion of profound academic issues, it is not 
certain that the activities of this project will continue. 
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Available Information 



The project produced a number of papers and reports. Chief among 
these are the Final Project Report to FIPSE, as well as a report on 
each participating discipline. Furthermore, several papers based on 
the work of various groups have been presented at professional 
organization meetings by project participants. Information about these 
papers can be obtained by contacting the project director or the group 
coordinators listed below, all of whom are faculty at SUNY/Plattsburgh 
in the departments of the discipline they represented. 
The specific reports are as follows: 



Biology Group Report, prepared by Bonnie Seidel-Rogol 

Foreign Language & Literature Report, prepared by Craig Sample 

History Group Report, prepared by Douglas Skopp 

Teacher Education Group Report, prepared by Nick Stupiansky 

Psychology Group Report, prepared by Henry Morlock 

Copies of all these reports can be obtained by contacting the Project 
Director: 



E. Thomas Moran 

Vice President for Academic Affairs 

SUNY/Plattsburgh 

Pittsburgh, NY 12901 

518-564-5402 
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WINTHROP COLLEGE 
South Carolina Higher Education Assessment (SCHEA) Network 
Purpose 

The South Carolina Higher Education Assessment (SCHEA) 
Network, a 47-member statewide consortium of South Carolina 
colleges and universities, was established in response to state and 
accreditor assessment mandates. Its primary purpose was to develop 
collaborative relationships among a group of colleges and universities 
largely unprepared for the challenges of assessment. Network 
organizers assumed that properly coordinated consortial arrangements 
and centrally provided technical assistance would enable members to 
initiate quality assessment programs faster and less expensively than 
they could achieve on their own. 

Innovative Features 

At the outset, there were few promising signs for an effective 
assessment consortium, given the limited resources committed in 
South Carolina to assessment, the fierce rivalries among sister 
institutions, and an unwelcoming attitude on the part of most higher 
education professionals towards state assessment mandates. 

When V/inthrop University emerged as leader of the consortium, 
some felt that it had few obvious qualifications to mount such an 
ambitious project. A relatively small institution, formerly a state 
women's college tucked up in a corner of the state, it had no 
distinctive mission or constituency. Yet several Winthrop admin- 
istrators and certain members of Winthrop's Psychology Department 
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foresaw the need to coordinate assessment assistance to member 
institutions statewide. They formed the Network Coordinating Center 
at Winthrop to provide organizational, informational, and technical 
assistance and mutual support services, and to collect and produce 
assessment resource materials. 

Evaluation 

The primary learning targets and beneficiaries of the SCHEA 
Network were the faculty and administrators of member institutions. 
Yearly evaluations were conducted of the Network's objectives, 
assessment activities of members, staff performance, conferences, 
workshops, presentations, consumer demands inside and outside of the 
Network, dissemination, and changes in SCHEA member participation. 

Five external evaluators, all prominent nationally in assessment, 
judged the SCHEA Network project and its activities very highly. 
They pointed to the consortium's statewide influence in assessment 
policy as evidenced by the Network's symbiotic relationship with the 
Commission on Higher Education, its effectiveness in promoting inter- 
campus collaboration, and its role as the assessment resource for 42 
higher education institutions in South Carolina. They dubbed it "the 
best state assessment consortium in the country, and one that other 
states should emulate." 

A th/ee-year evaluation data summary based on ratings from 
Network members and program records showed marked increases in 
Network membership and inter-campus collaborations; increased 
conference attendance; favorable attendee evaluation; consistently 
positive Advisory Board assessments of SCHEA, its activities, 
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objectives, services and staff; and skyrocketing gains over the three 
years in requests to the Network for technical assistance, biblio- 
graphies, and publications. 

Project Impact 

Originally, the project aimed to attract a core group of consortial 
participants from at least 12 institutions, who would generate 
momentum for participation across the whole state. In fact, in the 
three-year grant period, the SCHEA Network pulled together 42 
institutions, including all 33 public colleges and universities in the 
state, seven private institutions, and two state agencies. Beyond this 
organizational framework, Winthrop developed an exportable and 
practical primer, A Beginner's Guide to Higher Education Assessment , 
to help orient newcomers to the major issues, models, options, and 
obstacles to assessment. 

Today, hundreds of requests for assistance and materials from all 
50 states and six foreign countries far exceed the Network's ability to 
respond. Requests for on-campus workshops and needs assessments 
are especially difficult to answer. Network staff have annotated and 
printed over 700 assessment references, 120 instruments and methods, 
and eight assessment newsletters (The eXchanffe) . These materials 
constitute a lending library of assessment resource materials for 
SCHEA members. Winthrop's full-day Beginner's Guide Workshop 
has become a staple of AAHE's National Assessment Conference. 
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Major Insights and Lessons Learned 



Virtually every program associated with the SCHEA Network has 
reported positive impact on program quality. Admittedly, these 
programs refer to clear and substantial process benefits of assessment 
rather than student outcome data that may be forthcoming later. 

With a consortium, time and cost estimates seem to expand 
exponentially and are extremely hard to manage. The Network found 
that the nature of assessment made everything take longer than 
planned, whether it was designing, piloting, implementing, analyzing, 
evaluating or disseminating. 

Network staff found that most higher educators stand to benefit 
from undertaking assessment, whether they know it or not. Once 
misperceptions and myths about assessment were removed, SCHEA 
participants became more supportive and willing to put in long hours, 
believing that their effort would one day benefit their students. Even 
long-standing patterns of rivalry and mistrust among SCHEA institu- 
tions did not stand in the way of productive collaboration. The key 
strategy was to lure central campus leaders with the repeated theme 
that assessment would enhance the quality of programs. 

However, even the best programs can expect a hard core minority 
to oppose assessment, whether out of suspicion, fear, or bias. 
Winthrop's lesson from its own experience is similar to that of others, 
i.e., opposition is to be expected with any major reform, and it should 
not dissuade or impede assessment efforts. The Network credited the 
generous help of other FIPSE project directors in assessment who 
provided useful and innovative ideas that improved the efficiency and 
quality of the Network. 
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Project staff recommended that FIPSE fund: (1) more evaluations 
of assessment methods, especially of their validity; (2) more 
assessment consortia; and (3) more projects that consolidate and 
analyze the conflicts and redundancies among rapidly proliferating 
assessment-related laws, regulations, and guidelines. 

Project Continuation 

SCHEA Network activities have received sufficient support from 
the Commission on Higher Education, Winthrop University and 
membership fees to continue, despite the worst fiscal crisis in the 
state's history. Substantial interest in adopting the Network's South 
Carolina consortial model in North Carolina, under the sponsorship of 
the North Carolina Association of Colleges and Universities, has taken 
the form of exchanges of information, planning meetings and on- 
campus visits. 

During the project's second and third years, academic area 
networks of multi-campus interest groups were formed, to share 
assessment experiences and innovative strategies, and to plan 
collaborative activities. Begun with fewer than a dozen members in 
1990, there are now 14 mini-networks with over 200 members, one of 
the most promising legacies of the SCHEA Network. 

Additionally, SCHEA has begun establishing liaisons with 
assessment mandators and other groups influential in assessment 
policy and implementation. Through these liaisons SCHEA hopes to 
perform a beneficial consultative and advisory role. Last year a joint 
South Carolina Commission on Higher Education Staff/SCHEA Task 
Force developed recommendations for ways to implement the federal 
"Student Right to Know" and state "Higher Education Accountability" 
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acts in South Carolina. These recommendations have been adopted. 
In spring, 1992 SCHEA established an information-sharing relation- 
ship with the staff of the state and national Educational Goals Panels, 
and several SCHEA members have been nominated to serve on 
advisory councils to those groups. SCHEA nominations have been 
solicited by the Southern Association of Colleges and Schools regional 
accrediting agency to help strengthen their institutional effectiveness 
and assessment evaluator pool. In addition, SCHEA became a formal 
affiliate of South Carolina's most influential non-governmental higher 
education group, the South Carolina Council of Presidents. 

Since the SCHEA Network primarily represents the "troops in the 
trenches" in higher education assessment, these new liaisons may well 
afford earlier, more effective practitioner input to policy development, 
which will in turn encourage higher quality assessment programs 
throughout the state. 

Available Information 

The SCHEA Network has produced over 30 reports and other 
resource materials, including organizational documents (e.g., member- 
ship lists, committee charges, conference programs, and evaluation 
forms), project reports, and assessment-related publications. The 
Network will share these materials on a copy cost or maintenance fee 
basis. 
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The following are available: 

o SCHEA Network Publications List 

o A Beginner's Guide to Higher Education Assessment 

(Available in a workshop version and a condensed vei<*ion,) 

o The SCHEA Annotated Bibliography of Higher Education 
Assessm e nt Literature 

o The SCHEA Annotated Bibliography of Student Personnel 
Development Assessment 

o A Selected Bibliography of Learning Research Literature 

o A Critical Review of Student Assessment Options 

o The SCHEA eXchange Newsletter . Issues 1-8 

o The SCHEA Network Lending Library of Assessment 
Resource Materials 

o Assessing Assessment: An In-Depth Study o f the Higher 
Education Assessment Movement in 1990 (ACE-supported) 

o Sample Assessment Plans for Students' Personal Development 

o Final Report to FIPSE for the SCHEA Project 
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Please direct your requests to: 



Reid Johnson 
Office of Assessment 
210 Tillman Hall 
Winthrop University 
Rock Hill, SC 29733 
808-323-2341 
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COLI^GE AND UNIVERSITY TEACHING 



Assisting students who are having difficulty with basic courses has 
absorbed enormous amounts of institutional energy and proven a 
source of great frustration to faculty and administrators. At the 
University of California at Berkeley, the work of Uri Treisman has 
proven so successful in improving the mathematics performance of 
underprepared African-American students that it has been widely 
copied in a variety of circumstances and subject matters elsewhere. 
The "Medical Scholars Program" at the University of California at San 
Francisco exemplifies the adaptability, in this case to medical 
education, of Treisman's "Professional Development Program." At 
Salem State College, adaptation of a similar model, Supplemental 
Instruction, has improved student performance in a variety of arts and 
sciences subjects. 

These three programs make heavy use of collaborative learning and 
problem solving, and use advanced students as group facilitators. 
Their effectiveness is readily demonstrable. 

Of a different order of concern but no less vexing to those directly 
involved in teaching and learning are the problems of novice library 
users in gaining access to multiple sources of bibliographic assistance. 
The Ohio State University libraries have successfully addressed this 
problem by combining multiple sources of information into a single 
computer program employing simple commands and uniform screen 
formats. 



UNIVERSITY OF CALIFORNIA-BERKELEY 



The Professional Development Program 

Purpose 

In the late 1970's, Uri Treisman and colleagues at the University of 
California at Berkeley began a study to understand the dynamics of 
minority student performance in freshman calculus. A systematic 
review of Berkeley transcripts showed the magnitude of the problem: 
in a decade there was not a single year in which more than two ethnic 
minority students received grades of B- or better in first-term calculus. 
In his FIPSE proposal to remedy this situation, Treisman observed 
that freshman mathematics and science courses have all too often been 
burial grounds for the aspirations of minority students who enter 
college with the goal of majoring in engineering, or one of the sciences. 

An eighteen-month ethnographic study helped Treisman understand 
the various obstacles to minority student success at Berkeley. In 
particular, he found that the widely held conventional explanations for 
failure-lack of student motivation, lack of academic preparation, lack 
of family support, and low average socio-economic status— did not,' in 
fact, explain the phenomena he observed. He found, instead, a pattern 
of social and intellectual isolation which eventually led to students* 
demoralization, disorientation, and ultimately, their decision to leave 
the sciences. Many of these students, moreover, dropped out of the 
University. 

Many of the African-American students Treisman observed studied 
alone, and maintained a rigid separation of their academic and social 
lives. This pattern was in marked contrast to that of Chinese students 
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in Treisman's study, who typically created an academically-focused 
social group that served to support their aspirations for high 
achievement. In their informal groups, Chinese students discussed 
their academic work as well as such topics as interacting with faculty, 
the intricacies of getting a needed answer from an administrative 
office, financial aid, and so on, Treisman believed that by creating a 
rich mathematical environment in which such collaboration among 
students was natural, African-American and Hispanic students would 
be drawn to mathematics and would thrive. 

Treisman also observed the subtle and insidious ways in which 
special campus orientations and programs for minorities tended to 
marginalize the populations they served. In particular, he noted that 
for many minority students at Berkeley the locus of their campus 
identity was ethnicity rather than intellectual interests or professional 
goals. He came to believe that the resulting ethnic balkanization of 
the Berkeley student body impeded minority students* progress in 
mathematics-intensive majors. Thus, to address the underrepre- 
sentation of minorities in mathematics, it would be necessary to create 
viable and robust multiethnic student communities drav/n together by 
a shared interest in and affection for mathematics. 

Innovative Features 

In 1980, with support from FIPSE, Treisman established the 
Mathematics Workshop Program as a component of the Professional 
Development Program (PDP), an affirmative action effort of the 
University of California Academic Senate, Berkeley Division. The 
immediate and most visible goal of the project was the development of 
a cadre of minority students who would truly excel in calculus. The 
Workshop Program explicitly rejected remedial approaches to minority 
education. 
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Workshops of 20 to 25 participants worked together in small 
groups on problem sets specifically designed to deepen their 
understanding of concepts needed not only for their present course, 
but for later mathematics and science courses. These sessions were 
overseen by graduate instructors who did not lecture or repeat class 
material, but turned questions back on the group to help students 
understand the material for themselves. This approach was a 
substantial departure from the individualized tutoring, self-paced 
instruction, mentoring, and study skills courses which constituted the 
pedagogical armamentarium of institutionalized minority programs. 

Evaluation of the Original Development of PDF 

According to the evaluation of the first seven years of PDP, 
students did indeed excel. The results are unparalleled: 56% of the 
African-American workshop students earned a grade of B- or better in 
first-year calculus, compared to 21% of the non-workshop African- 
American students. Beyond this, significantly fewer workshop 
students dropped out of calculus than non-workshop students (3% 
versus 25%) and four times more of these students graduated in math- 
based majors (44% versus 10%). These results persisted even when 
differences in pre-college preparation for mathematics were taken into 
account. 

African-American students who attended the workshops and whose 
MSAT scores suggested they were at the greatest risk of failure 
outperformed the non-workshop African-American students at lowest 
risk of failure. In other words, weakly prepared students gained as 
much or more than students entering Berkeley with strong academic 
backgrounds. Further, African-American workshop students persisted 
to graduation at rates comparable to those of the campus as a whole, 
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while African-American non-workshop students dropped out at 
substantially higher rates. 

Project Impact 

The Mathematics Workshop had a dramatic effect both on 
mathematics performance and on the persistence rates of participating 
students at Berkeley. Based on these successes, FIPSE again funded 
PDP in 1986 to assist 20 other campuses across the country to set up 
local adaptations of the project. PDP staff worked with faculty and 
administrators from colleges and universities to customize the project 
appropriately for each new setting. Many of the original twenty 
dissemination sites succeeded in creating effective programs: an 
evaluation of student performance at several of them showed the 
following results: 

o At the University of Texas-Austin, where the program is known 
as the Emerging Scholars Program (ESP) , the workshops are 
regular departmental offerings-effectively an alternative to 
traditional discussion sections. There are presently five ESP 
sections serving approximately 100 students per semester, 
three-quarters of whom are African-American or Hispanic. The 
program also targets white stu-dents from rural Texas. Results 
of the program have been dramatic. Over 85% of all ESP 
students routinely earn grades of "A" or "B" in calculus. (To 
place this in an historic context, fewer than one-third of non- 
ESP African-American and Hispanic students who took calculus 
in the last five years received grades of "A" or "B." Only half of 
those students with Math SATs over 600 received such grades; 
in contrast, 90% of the minority ESP students with such scores 
have earned grades of "A" or "B."). In 1988, there were fewer 
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than 10 minority math majors at UT Austin. In Spring 1992, there 
were 113 minority mathematics majors, many of whom were ESP 
alumni. 



o At Rutgers, the State University of New Jersey, students enroll in 
a six-credit-hour calculus class called EXCEL; the usual calculus 
course is four credit-hours. One section of EXCEL is housed in the 
College of Engineering, and another is housed at Douglass College 
for Women. Thus far, 75% of EXCEL calculus students have 
earned a grade of "A" or M B ,! as opposed to 41% of all students in 
the regular calculus classes. 

o City College of New York (CCNY) is an urban commuter college; 
two-thirds of the entire student population is African-American or 
Hispanic. Students in the CCNY workshop program participate in 
six hours of intensive mathematics laboratories as a supplement to 
their basic calculus class; four hours of this lab time is overseen by 
advanced undergraduate students. 88% of participating students 
have received grades of "B" or better in calculus. 

o At the California Polytechnic State University at San Luis 
Obispo, the Emerging Scholars Program (ESP) offers workshop 
laboratory sections in precalculus and calculus. These labs are 
attached to particular lectures and are led by advanced 
undergraduates who have expressed an interest in teaching. 
Consistently over the four years of the program, 90% of the 
students have achieved grades of ,f C M or better in calculus; typically, 
two-thirds of non-ESP students make such grades. In 1988, the 
year of the pilot, 85% of ESP students earned grades of "A" or "B," 
while only 18% of students not participating in ESP labs earned 
similar grades. 




o The California State Polytechnic University at Pomona runs a 
workshop program for minority students enrolled in the College of 
Engineering and the College of Science. The program serves more 
than 100 minority freshmen a year. A recent retrospective 
longitudinal study (conducted by Marty Bonsangue and supported 
by the National Science Foundation) of minority workshop 
participants from 1986 to 1991 found that workshop students 
scored, on average, six-tenths of a grade point above their 
classmates in calculus. Bonsangue found that participation in the 
workshops had a powerful effect on students* performance and 
persistence in subsequent mathematics and science coursework, as 
well as on graduation in a technical major. 

o The Medical Scholars Program (MSP), an adaptation of the 
workshop, was established with FIPSE support at the University of 
California at San Francisco School of Medicine (described on page 
77). Minority student failures in physiology and other first year 
courses declined significantly; the school's overall pass rate on the 
National Boards improved; and pre-medical school achievement 
ceased to be a significant predictor of minority and non-minority 
students* scores. 



Evaluation of the P DP DiflsPTnin ation 



An extensive summative evaluation was conducted of each of the 
campus pilot programs and student performance, using appropriate 
comparison groups. Specifically, it examined the relationship between 
background characteristics (e.g., race, sex, SAT scores, high school 
rank in class, etc.) and achievement in math and science. A qualitative 
observer's report was also completed to provide faculty at other sites 
with a description of key features of the program, how it operates 
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internally, how it may be adapted, and how it appears to its 
participants. 

Project Continuation 

There are currently over 100 mathematics departments offering 
students a Workshop/Emerging Scholars" program. In 1991, 
Treisman received a grant from the National Science Foundation 
(NSF) to create the Alliance for Minority Participation in Mathematics 
(AMPM), which will expand to at least 200 the number of mathematics 
departments offering such programs. 

The program has expanded both to the upper-division and pre- 
collegiate levels. The Summer Mathematics Institute (SMI), variously 
funded by the Sloan Foundation, the Dana Foundation, and the 
National Science Foundation, is an immersive six-week residential 
summer school, principally for ESP alumni, designed to enable them to 
make an informed choice about graduate work and careers in the 
mathematical sciences. In fall 1993, approximately 20 SMI alumni will 
begin work in mathematics at elite graduate programs. 

At least ten mathematics departments have created versions of ESP 
for high school students and teachers. These initiatives, most often 
called "C-cubed" programs, have received generous support from the 
National Science Foundation and from local school districts. 

Several successful sites have emerged as secondary dissemination 
sites, helping institutions in their region to learn from their work. 
Moreover, the process of dissemination has led to substantial 
improvements in the model, some of which have influenced the shape 
and direction of the original Berkeley program. 
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The Workshop Program itself has undergone many transforma- 
tions. Most important is the shift from adjunct status, i.e., as a 
voluntary add-on program, to becoming an integral part of a math 
department's offerings. In some institutions, the Workshop Program 
has been a catalyst and model for the reorganization of departmental 
courses. Currently, there are many departments seeking to utilize the 
Workshop model as a vehicle for bringing about curricular change. 
The increased class time and the focus on student community make it 
a natural environment for curricular experimentation. Finally, the 
Workshop approach has become a highly visible example of faculty 
leadership in such problematical areas as multiculturalism, academic 
culture change (i.e., resetting the balance among research, teaching, 
and service), and quality management. 

Recognition 

PDP's project director, Uri Treisman, was honored with the 
Charles A. Dana Award for outstanding achievement in American 
higher education. The Dana Foundation then awarded Berkeley 
approximately $1 million for the creation of the Charles A. Dana 
Center for Mathematics and Science Education to continue to 
disseminate Workshop Programs. In 1989, Newsweek selected 
Treisman as one of 25 Americans on the leading edge of innovation, 
one of three in education. In 1992, he won the MacArthur Fellowship 
Award for his pioneering work in increasing minority participation in 
mathematics. 

The California State Legislature awarded $500,000 to create PDP 
workshops on each of the nine campuses of the University of 
California. The National Science Foundation sponsored a Chautauqua 
short course for math and science faculty on increasing minority 
participation in math-based disciplines. A minority student and 

74 



alumna of PDP at Berkeley was named a Rhodes Scholar--the first 
woman, first African-American and first engineering student on 
campus so honored. 

Available Information 

A FIPSE-spon8ored lecture that describes the project, Academic 
Perestroika : Teaching. Learning and the Faculty's Role in Turbulent 
Times , is available upon request from: 



Uri Treisman 

Dana Center for Mathematics Education 
National Sciences Annex 1202 
Austin, TX 78712 
512-471-6190 



UNIVERSITY OF CALIFORNIA AT SAN FRANCISCO 
SCHOOL OF MEDICINE 



The Medical Scholars Program 



Purpose 

Nationwide, minority students and women are underrepresented on 
the faculties of medicine and the basic sciences. An American 
Association of Medical Colleges study indicates that fewer than 2% of 
the nation's medical school faculty are from the underrepresented 
minorities. The UCSF Medical Scholars Program (MSP) was launched 
as a direct response to the need for minority academicians. 

A secondary and more immediate aim was to reduce minority 
attrition and course failure of "at-risk" students. MSP is based on the 
Professional Development Program pioneered in mathematics at the 
University of California-Berkeley, using small study groups or 
workshops to promote achievement among minorities. Applying the 
PDP model, MSP used peer groups and preceptorships to improve 
learning in pre-clinical courses. 

Even though MSP was initially labelled a minority program, it was 
designed to be open to all first-and second-year medical students, and 
non-minority students did fully participate. It made the uncommon 
assumption that students entering the program had the ability to 
excel, regardless of test scores, and its job was to aid in that 
achievement, MSP's intent was to be prospective and preventive 
rather than remedial. 
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Its student-led workshops met weekly for two hours to work on the 
key organizational and conceptual skills necessary to master vast 
amounts of medical information* To do this, student groups of 20 to 
25 used a series of problem sets to quiz and test each other. These 
worksheet problems were constructed so that students had to forge 
solutions collaboratively and use a variety of skills to answer questions 
posed in physiology, anatomy, and biochemistry courses. Ultimately, 
the problems led students to examine the relationship between their 
courses and clinical material by applying basic concepts to abstract 
situations. 

The workshops performed obvious social functions as well. The 
encouragement and support they provided eased the transition to 
medical school, helped students build friendships, and taught them 
survival skills. 

In the second year, students participated in a special study group 
review to prepare them to pass the National Board of Medical 
Examiners Part 1 test, a certification examination taken after pre- 
clinical studies. 

Beyond supporting coursework and preparing for the National 
Boards, the program oriented its students to research careers through 
a series of informal "science chats" with faculty and a quarter-long 
research elective. During summer internships, students participated in 
basic social science research in laboratory settings, in local commu- 
nities (AIDS and sexuality), and overseas (diabetes in Italy, cardio- 
vascular disease in Spain). 
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Evaluation 



Historical information from medical school student records was 
used as a quasi-experimental control for the results observed during 
the operation of MSP. Data from MSP's three years (1987-89) were 
compared to data for the three years prior to MSP. The latter 
indicated that minority students enter UCSF with a slight academic 
disadvantage: between 1983 and 1986 minority students failed courses 
and the National Boards at rates out of proportion to their numbers ii 
class and strongly correlated with their academic disadvantage. 
Failures among first-year minority students were most pronounced in 
physiology. In 1983, for example, African-American and Latino 
students constituted 18% of the enrollment in physiology but 54% of 
those who failed the course. 

Significantly, in the program's three years, none of the minority 
students enrolled in MSP physiology workshops failed, and MSP 
students passed all of their first-year basic science courses. They 
achieved this even though their mean undergraduate GPA and their 
combined MCAT scores were significantly below those of non-MSP 
students. 



Project Impact 

In its first three years, MSP served 164 students, 93 white and 7 
non-white. It can claim three strong outcomes: minority failures in 
pre-clinical courses declined; the school's pass rate as a whole im- 
proved on the National Boards; and pre-medical school achievement 
ceased to be a significant predictor of minority and non-minority 
students' exam scores. 
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Over the years, the number of minority students participating in 
summer research doubled. One student published her findings in the 
New England Journal of Medicine and another won a year-long 
research fellowship at the National Institutes of Health. 

Annual interviews with workshop students, leaders and faculty 
members suggested a school-wide effect of two kinds: MSP led 
minority students to believe that careers in academic medicine were 
possible for them, and it showed them the value of group study and of 
seeking academic help. Two unanticipated benefits were also noted: 
the workshops helped identify early those students having academic 
difficulty, and students gained valuable experience as workshop leaders 
and teachers. 

Maior Insights and Lessons Learned 

Medical school in the United States is reputedly a stressful and 
demanding experience. As viewed by its students and faculty, MSP 
positively altered the environment of UCSF without compromising 
quality~in fact, student performance improved. This holds important 
lessons for changing the learning climate in other medical schools 
further. MSP validates the Professional Development Program at UC- 
Berkeley, which originated this approach to improving minority 
performance, by demonstrating that PDP can be translated from math 
to the medical school curriculum without losing its impact on learning. 

Project Continuation 

Institutional policies were adapted to support MSP as a student- 
directed initiative, now funded through the Dean's Office. That 
decision was made on the basis c the evaluation data collected for the 
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program, including feedback from student participants. Even after 
FIPSE funding ended, the numbers of students served by the program 
continued to increase every year, and now encompass over 70% of the 
first-year class. 

A dissemination grant from FIPSE was used to prepare workbooks 
from the collection of learning-group worksheets. These workbooks 
were distributed free of charge to medical schools around the country. 

In 1991, Columbia University College of Physicians and Surgeons 
(P&S) created the Higher Achievement Program (HAP), modeled on 
MSP and motivated, in part, by a marked increase in the number of 
minority students entering in the first-year class that fall. Past 
experience of minority students at P&S suggested that these new 
entrants would be at high risk of failure. For example, in 1988, three 
of twelve (25%) entering minority students had to repeat their first 
year coursework, and two of sixteen (13%) minority students entering 
in the 1990 academic year were forced to repeat. Faculty and students 
agreed that some intervention was needed to ensure the smooth 
transition of students from undergraduate to medical school study. 

In its first year of operation, HAP offered workshops in anatomy, 
physiology, neurosciences, cell biology and physiology. In addition, 
special sessions were offered to prepare students for practical exam- 
inations in anatomy and cell biology. The program was remarkably 
effective: the number of students failing courses declined, even though 
the number of students at risk for failure had increased. None of the 
minority students entering in 1991 was asked to repeat the year. A 
full evaluation of the program will be conducted in the future, but 
faculty and students were encouraged by the results of HAP's first 
year. 
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Available T nformation 

Copies of the final report submitted to FIPSE and of the MSP 
workbook used in the study groups may be obtained from: 

Emilie Osborn 

Associate Dean of Student and Curricular Affairs 
University of California at San Francisco School of Medicine 
Room S221 

San Francisco, CA 94143 
415-476-0714 



or 



Mindy Fullilove 

Associate Professor of Clinical Psychiatry and Public Health 
722 W. 168th Street, Box 29 
New York, NY 10032 
212-740-7292 
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THE OHIO STATE UNIVERSITY LIBRARIES 

4 

The Gateway To Information 



Purpose 

Librarians are universally familiar with the novice researcher who, 
having been set a writing task, proceeds directly to the card catalog, 
jots down the titles available on his topic, checks out the ones he can 
find on the shelf, and considers his bibliographic effort complete. 
Periodical indexes, abstracting services, biographies, almanacs, govern- 
ment documents, and technical reports might as well not exist: the 
library's arsenal of information mystifies and intimidates most stu- 
dents, and only the adventurous or the experienced ever benefit from 
it. 

The Gateway to Information, a microcomputer program designed 
and developed in the course of this project, guides students to find, 
evaluate and select the materials that meet their research needs. It 
functions as a front-end to the online library catalog and CD-ROM 
databases, guiding users to create a search strategy that includes print 
and computerized information. It enables users to access and integrate 
the content of online catalogs and CD-ROM databases, and encourages 
them to seek varied sources of information and to evaluate the infor- 
mation they find, 

In BQvgtjye Features 

Most libraries use computer technology to inform the public about 
general library services or to conduct specialized searches of com- 
mercial databases or of CD-ROM-based indexes. The libraries that 
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teach all-encompassing search strategies do it with lectures and 
printed materials, and only a few offer an organized information- 
seeking approach that integrates different sources. The Gateway's 
uniqueness lies not only in its ability to accomplish the latter, but in 
that it guides the user to identify, locate, and evaluate the appropriate 
materials. 

The Gateway provides access to 3] dictionaries; five general and 82 
subject encyclopedias; 70 periodical indexes (including 16 on CD-ROM); 
28 biographical sources; 35 book review indexes; and three general and 
12 specialized statistical sources of local, national and international 
data. The Gateway presents all these sources on screens that have a 
common screen layout. Electronic resources are accessed by simple 
commands, thus making it unnecessary for the user to learn a number 
of different protocols. The Gateway software includes HyperCard 2.0, 
MAC/TCP, and Mitem-View, and runs on Apple Macintosh NCX com- 
puters, which are connected to the campus computer network. 

Evaluation 

In addition to being observed and interviewed by staff, The 
Gateway users filled out over 2.000 evaluation forms over the course of 
the project. This feedback was used to modify and refine The 
Gateway, and the response, which had been positive from the begin- 
ning, grew more enthusiastic with each improvement. Students were 
particularly pleased with the ease of access and the abundance and 
appropriateness of information provided by The Gateway. No attempt 
was made to compare the quality of research carried out with the 
assistance of The Gateway to that accomplished by traditional means. 
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Project Impact 



Originally designed for use by lower division undergraduates, with 
each refinement The Gateway attracted progressively more sophisti- 
cated researchers, so that at present 32% of its users are graduate 
students, faculty and staff. Faculty are generally pleased with the 
effects of The Gateway on student work. The program has attracted 
interest at other institutions, and conversations with various market- 
ing companies are currently underway. 

Recognition 

The Gateway project has received two additional grants from the 
Department of Education's College Library Technology and Coopera- 
tion Grants Program, and one from the William Randolph Hearst 
Foundation. 

Project Continuation 

The Gateway was originally available at only four library work- 
stations. By the end of the project these had grown to 24, and 
eventually all 60 public workstations at the University Libraries will 
have The Gateway capability. The narrative is constantly being 
expanded and revised, and special subject sections are being added, 
such as Communication, Business, Women's Studies, and University 
Archives. The University's continuing support of this project may 
justly be attributed to the enthusiastic reaction to The Gateway on the 
part of students, faculty and staff. 



Available Information 



Inquiries about The Gateway should be addressed to: 
Virginia Tiefel 

Office of Library User Education 
The Ohio State University Library 
1858 Neil Avenue Mall 
Columbus, OH 43210 
Telephone: (614) 292-6151 
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SALEM STATE COLLEGE 

Improving Teaching and Learning in Introductory 
Arts and Sciences Courses 

Purpose 

Disturbed by the frustrations of both faculty and students in 
certain freshman and sophomore courses that produced high percent- 
ages of student failures and withdrawals,^he Director of Academic 
Advisors turned to the Supplemental Instruction (SI) Program model 
developed by Deanna Martin of the University of Missouri, Kansas 
City. The model seemed to hold promise not only for reducing failures 
in these "high risk" courses, but also as a vehicle for faculty develop- 
ment, making the courses more rewarding for faculty to teach. 

The SI program sets up a regular schedule of out-of-class sessions 
in which groups of students work together to master course content. 
Leaders for these groups are drawn from among students who have 
3 'essfully completed the courses that employ this model and have 
bee x nominated by faculty to serve in this role. The program thus 
focuses on the academic content of specific courses, helping students 
develop learning strategies tor that particular content. 

P roject Activities 

Salem State expanded the model by encouraging faculty teaching 
these courses to help interview, select and train the SI leaders. 
Faculty also attended monthly seminars where they shared their 
frustrations and discussed ways of dealing with the classroom 
problems they faced in common. Sessions ranged in format from guest 
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speakers to seminar members presenting ideas that worked, and at 
least once each semester the faculty group and the SI leader group met 
together. Thus faculty became familiar with the program and involved 
with the student leaders as colleagues participating in a common 
enterprise. 

The trained student leaders attended all sessions of their faculty 
members course and led voluntary supplemental sessions of two or 
three hours per week. At first, these sessions were likely to be 
attended by the better students who wished to improve their course 
performance, but the more marginal students eventually began to 
attend. (Faculty never knew until the course was over which students 
had attended these sessions.) Faculty were encouraged to meet with 
the SI leader once a week to answer questions or work together on 
instructional problems. Although these meetings were for the express 
purpose of having faculty help the SI leaders, nearly all faculty found 
that the leaders* reports on sources of student difficulty helped them 
adjust their instructional emphasis and strategies. 

Innovative Features 

While a number of out-of-class group study strategies such as SI 
have demonstrated their success, the kind of faculty involvement 
featured in this project is unusual. Faculty participation in training SI 
leaders and in all-faculty seminars turns an academic support activity 
for students into a faculty development effort as well. In the process, 
faculty begin to find solutions to pedagogical problems and to adopt a 
new attitude toward teaching courses in which student performance 
and intellectual challenge are unsatisfying. 
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Evaluation 



Project directors kept detailed records of the course performance of 
students in all sections that offered SI support over a three-year 
period. The average grades of students participating in SI instruction 
were half a grade higher that the average of those not participating. 

To address the argument that SI participants might be intrinsically 
better students than non-participants, grades in sections of the same 
course that offered no SI support were compared with those of SI 
participants. The results showed an average grade differential quite 
similar to that between SI and non-SI students in the same course. 

Student attrition in these courses also declined. Thus, between the 
support provided by the SI leaders and the instructional changes 
introduced by faculty as a result of their program participation, a 
significant number of the courses moved out c r the "high risk" 
category. 

Faculty reaction to the experience was measured by a ques- 
tionnaire administered before and after participation, and further 
assessed through voluntary interviews of half an hour to an hour at 
the conclusion of each year's activities. Results, both statistical and 
anecdotal, are quite encouraging. Faculty felt inspired by having the 
SI leader in their classes and found the feedback from the leader 
helpful. They felt that they had become better teachers as a result of 
the experience and were most satisfied with teaching introductory 
courses. They learned a great deal from the seminar about teaching 
more effectively. 
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TTnflntiripftted Problems 



Student SI leaders sometimes had trouble understanding the 
difference between tutoring and SI. The purpose of SI is to help 
students support each other in learning how to learn and n mastering 
course material, rather than to have the SI leader reteach the 
material. 

Faculty sometimes had the same problem. Part of the problem lay 
in getting faculty to commit the two full days necessary to obtain a 
thorough understanding of the SI technique, which made for some 
difficulties in their supervision of the SI leaders in their courses. 

Major Inp ighta «p d Lessons Learned 

SI works not only to impro ve student performance and reduce 
course attrition, but with the intimate involvement of faculty it can 
result in greater instructor satisfaction and pedagogical renewal. 
Faculty discover that their problems are shared by those in a variety 
of other disciplines and become more willing to work on them with the 
moral support of others in similar situations. 

The experience of being an SI leader can have substantial impact 
on student development, building leadership skills as well as academic 
ones. In at least eight cases, the experience resulted in students 
choosing a teaching career they had not previously considered. 
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Project Co ntinuation 



Despite the unusually large budget reductions experienced by Salem 
State along with other Massachusetts institutions, campus funding 
continues and the program is quite active. 

Available Information 

The project produced several videotapes that will be of interest to 
others contemplating introduction of such a program. Several small 
pamphlets summarize the impact of the project on students and 
faculty. 

These items and general information about the program are 
available from: 

Douglas A. Larson 

Department of Accounting/Finance 



Sherrin Marshall 

U.S. Department of Education 

Fu.id for the Improvement and Reform of 

Schools and Teaching 
555 New Jersey Avenue 
Washington, DC 
202-219-1496 



Salem State College 
Salem, MA 01970 
508-741-6664 



or 
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CURRICULUM AND TEACHING IN THE DISCIPLINES 

All nine of these programs have as a common feature the creative 
adaptation of modern electronic technology, ranging from computers to 
programmable calculators to TV satellite transmission. The uses of 
this technology vary from simulations to self-instruction to practice, 
but all reflect pedagogical imagination applied to widely experienced 
classroom problems. 

Six of the nine projects are in the sciences, and of the thre* non- 
science projects, two are in fields whose theory is strongly based in 
mathematics. It may be that the sciences lend themselves better to 
the kinds of instructional adaptations of technology that FIPSE has 
promoted; it may be that the teaching of science has been a matter of 
special concern to the disciplines in recent years; or it may be that 
successful short term learning outcomes are more readily demonstrable 
in the sciences. Whatever the reason, the predominance is notable. 

Several of the projects are self-consciously based in contemporary 
learning theory. Thus they sequence learning differently from what 
has been the custom in the field, strive less for coverage and more for 
student independent discovery and understanding of basic principles, 
and promote active learning. Because in many cases microchip tech- 
nology eliminates the drudgery or even the impossibility of complex 
calculations, changes are occurring in what gets taught. 

All of these projects place the student at the center of the learning 
experience. The lecture has not become obsolete, but these strategies 
have the effect of reducing the faculty role in mediating learning and 
make it much more a matter of student effort and discovery. If stu- 
dents do indeed understand better and retain longer what they 
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discover themselves, the argument for wide adoption of such strategies 
is strengthened. Fortunately, the specific curricular and instructional 
materials ds sloped through these projects have been demonstrated to 
be adaptable at many different kinds of campuses. Nearly all project 
directors report wide dissemination of their work, with several 
reporting controlled evaluations of their materials on campuses other 
than their own. 
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CLEMSON UNIVERSITY 
Investigative Learning: A Plan for Laboratory Education 



Purpose 

Instructors of introductory biology for non-majors completely 
revised the laboratory portion of the first semester of the course to 
focus on the scientific investigative process. A set of student-designed 
and -executed experiments has replaced the familiar "cookbook" 
exercises for traditional laboratories. The new format aims to give 
students a better sense of the process of scientific investigation and, 
through extensive writing and oral presentations, to improve their 
communication skills. The project directors wished to accomplish 
these goals without negatively affecting student performance in the 
lecture course, which the traditional laboratories help to support. 

Project Activities 

Instructors have completely redesigned the laboratory portion of 
the course, substituting a computer simulation of commercial aqua- 
culture and a series of student-designed "wetlab" experiments for 
traditional laboratory exercises. In support of the new approach, the 
project directors created extensive new instructional materials: 

1) FISH FARM, a computer simulation challenging students to 
determine uie economically optimum culturing conditions for a 
hypothetical new fish hybrid. 

2) "Methods modules," consisting of: a videotape that demon- 
strates standard laboratory techniques; a student manual of step- 
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by-step processes to be followed in designing, carrying out and 
reporting an experiment; an instructor's guide; and a preparatory 
guide. 

3) A writer's guide and writing exercises, produced in collaboration 
with members of the English Department. 

Students begin the semester with the computer simulation. Teams 
of three or four students experimentally determine the best values for 
each of five culturing variables. The teams then test their conclusions 
by simulating five years of commercial operation under their predicted 
optimum conditions. Students using the same unknown fish compete 
against one another for the most profitable results. Finally, each 
student writes a FISH FARM report. The emphasis in grading these 
reports is on correct presentation and interpretation of the results, not 
on profit. 

After completing FISH FARM, students begin the "wetlab" work. 
The teams formed for FISH FARM view videotapes demonstrating 
laboratory techniques. In consultation with their instructor, they 
devise an original investigation using one or more of the techniques 
and design an experiment. They present an oral research proposal to 
the class and modify their plans based on feedback from their peers. 
They then submit a materials list to the prep room, perform the 
experiment, and present their results orally to the class. Each student 
submits a written report. In the course of the semester, each team 
carries out three experiments. 

Innovative Features 

Laboratory courses in the sciences usually have the stated goal of 
improving students' ability to think scientifically by doing experiments 
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and interpreting the resulting data. A secondary purpose is review 
and reinforcement of content covered in course lectures. All students 
perform the same exercises following highly detailed directions. They 
know in advance the outcomes they are to achieve, and they are 
required to explain these "correct" outcomes rather than think about 
what experiments to perform or how to perform them. 

In the investigative laboratories designed and implemented through 
this project, students devise and report on their own experiments, 
following procedures much more like those of scientific investigators. 
In close collaboration with others, they define the question, structure 
an investigative process, choose appropriate investigative techniques, 
carry out the experiment, and report results in written and oral form. 
Grades are based on the quality of the process and of the written 
report rather than on a series of content-driven quizzes. 

Such a laboratory program is rare because it is difficult to imple- 
ment. The low skill levels of students, the unpredictability of lab 
material requirements, the lack of time for thoughtful development of 
experiments and the additional demands on the creativity and re- 
sourcefulness of instructors all militate against this strategy. 

Evaluation 

The project aimed to improve the knowledge and skills of students 
about the experimental process, and to enhance their writing without 
sacrificing mastery of subject matter. The design of the evaluation 
was simplified by the fact that a number of lab section instructors, all 
of whom were graduate assistants, taught both investigative and tra- 
ditional labs. 
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This allowed the separation of students into three groups: those in 
investigative sections taught by instructors who also taught traditional 
sections; those in these same instructors' traditional sections; and 
those taught by instructors who taught only traditional sections. 
Mean lecture examination scores for the three groups showed no 
significant difference, demonstrating that students in investigative labs 
were able to master course content as well as those in the traditional 
labs, one of whose purposes is to reinforce lecture content. 

On the other hand, students in the investigative sections did not do 
better on test questions about process skills or the nature of science 
than those in traditional sections. Neither pre-test nor post-test 
results showed any significant difference among the three groups. A 
test of experimental design ability disclosed investigative superiority in 
some areas, but no significant difference in overall scores. Nor was 
there much difference on a test of writing, but the fact that the results 
of all groups were worse on the writing post-test than on the pre-test 
suggests that students did not take the post-test seriously since they 
knew it would not be part of their grades. 

Student opinion tended to be quite favorable toward the individual 
parts of the investigative structure but negative toward the whole, 
largely on the grounds that it was much more demanding and time 
consuming than the traditional sections. On the other hand, positive 
response to the wetlab manual, to FISH FARM, to the writing guide 
and the videotapes as well as to the freedom to devise experiments 
ranged from 50-75%. A majority agreed that they were more confident 
in their abilities to analyze problems scientifically, design experiments, 
analyze data and present their conclusions orally and in writing. The 
majority did not agree that they would have learned more about 
science in a traditional section. 
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Although the more negative global response clearly focused on the 
work demands, perhaps it also resulted from what beginning students 
believe to be the nature of learning. Because the lab sections had 
more to do with understanding a process than with memorizing facts, 
students were not clear about just what it was they had learned. 

Laboratory instructors were uniformly enthusiastic about the 
project from the beginning, and rated it highly with regard to their 
perceptions of student growth in the skills the course aimed to impart. 
They were concerned about the investigative students' understanding 
of the experiments but were equally disappointed in tlie traditional 
students' comprehension of the lab exercises. All but one thought the 
investigative labs required a higher level of effort, but that did not 
dampen their enthusiasm for the method. 

Although the laboratory preparators initially found that the 
investigative labs made their task more difficult, once they developed 
new ways of organizing their work they felt quite comfortable, A 
minority of the faculty remain concerned that the review of lecture 
course content provided by the traditional labs has been lost. 

Project Imp act 

The project has been well received among faculty and admin- 
istrators at Clemson and seems likely to be fully institutionalized. 
Because teaching the investigative sections is more demanding, two 
such sections are considered a full load for teaching assistants, as 
opposed to three sections of the traditional laboratories. The need to 
increase the number of TA's is the only barrier to full adoption of the 
investigative mode. 
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The project's impact at Clemson will be increased by some 
additional funding. In summer of 1992, the investigators received a 
National Science Foundation (NSF) grant to extend the investigative 
format to the second semester introductory laboratory course. (FIPSE 
funding only influenced the first semester course.) The funding will 
allow development of new exercises which are sufficiently flexible to 
make an investigative format worthwhile. 

In a broader sense, the most important result of the project is that 
this approach to laboratory instruction is practicable even in an 
introductory course for non-majors. Such a course precisely fulfills one 
major purpose of a general education science course: to give students a 
sense of the nature of the scientific enterprise. 

This project has generated interest on the part of teachers at other 
institutions. Portions of the course and course materials have been 
adopted by a local community college. Presentations at professional 
meetings have been well received, including a detailed presentation at 
the conference of the Association for Biology Laboratory Education. 
An NSF-funded workshop in the summer of 1990 drew 30 participants 
who, after five days, agreed unanimously that the work was useful to 
them and that they planned to implement investigative labs at their 
own institutions. Many have in fact done so, sor using materials 
from this project. 

The project has received two grants from NSF, one for the 
workshop mentioned above and another for video and computer 
equipment. A 1992 NSF grant will allow extension of the investigative 
format to the second semester course. 

At the invitation of a commercial publisher, one of the project 
directors is producing a lab manual. The other project director is 



100 

M9 



publishing FISH FARM with the same company. 

Major Insights and Lessons Learned 

In addition to being practicable, investigative laboratories are more 
interesting to teach than traditional ones. They cast instructors in the 
rde of research mentors, teaching students the characteristics of a 
good scientific investigation and then helping them apply those 
principles to the work they want to <?.o. Investigative laboratories can 
also offer interesting surprises when a student experiment turns up 
unexpected results. 

Available Information 

Project directors will send a general description of their im- 
plementation plan, including the lab syllabus and a summary of 
evaluation results. The laboratory manual and computer program 
are currently being prepared for commercial publication. Request 
materials from: 

Robert J. Kosinski 
Biology Program 
330 Long Hall 
Clemson University 
Clemson, SC 29634-1902 
803-656-3830 
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Revitalized Undergraduate Mathematics with Symbol 
Manipulating Graphics Calculators 

Purpose 

High-level programmable Hewlett-Packard calculators may afford 
students, for the first time, real graphical, numerical and symbolic 
computing power that is reliable and portable. But most freshmen and 
sophomores in undergraduate mathematics are still computing with 
pencil and paper, even though new computational devices are available. 
The goal of the project was to integrate these graphics calculators 
effectively into she service mathematics courses for science and 
engineering: single-variable calculus (two courses), multivariate 
calculus, differential equations, linear algebra and statistics. 

Innovative Features 

Clemson's ambitious experiment took aim at the actus 1 character of 
undergraduate mathematics instruction and learning. Reshaping the 
service courses for science and engineering students with the use of 
"supercalculators" would give them the power to do symbolic algebra 
and sophisticated graphics, and to use interactive operating modes in 
class. Instruction would move away from stressing manipulation skills 
and toward enhancing conceptual understanding. 

Prototype and pilot versions of the six courses were designed and 
tested with the following questions in mind: Where is calculator use 
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appropriate? How does calculator use affect efficient coverage of 
course material? Can calculators enhance conceptual understanding 
and allow now topics of study? 

Hewlett-Packard lent the project 95 graphics calculators, and 
Clemson's administration provided another 85. Students made almost 
daily use of them for both classwork and homework, which allowed 
them to explore and experiment as they studied core theory and 
methods, to get immediate feedback from instructors, and to interact 
in class in new ways. 

Evaluation 

No objective measures of student learning, such as comparison of 
group performance on common tests, have been undertaken. Match- 
ing student groups by ability, math background, and skill levels was 
very difficult. Because the powerful calculators allowed theoretical 
questioning as well as computational questioning, students taught in 
the traditional mode were found to be at a great disadvantage when 
tested without calculators at their side. 

Instead of using objective measurements, faculty observed classes, 
evaluated course materials (syllabi, handouts, and tests), and inter- 
viewed and surveyed 969 students in 39 classes about mathematical 
understanding, problem solving and exploration using the graphics 
calculators. 

Project Impact 

The claims of success for this project, then, are not based on 
comparative learning scores. Rather, project staff claim that tha 
calculators, more than anything else, changed not only what they 
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taught and how they taught it, but how and what they tested, and 
how students learned. The unique dynamic introduced into the 
learning process by these calculators "encouraged the students to 
learn-and the faculty to teach--the concepts and methods in a more 
active, constructive environment from analytical, graphical and 
numerical perspectives." 

The integration of the graphical and numerical solve features of 
these devices has enabled students to make important visual and 
numerical connections to the analytic presentations in the texts. The 
technology, then, removed the traditional routine computation and 
permitted more thinking about the underlying mathematical concepts 
and theories. 

Having observed students in class, faculty claimed that the way 
students learned was fundamentally changed by the integration of 
graphics calculators-that they developed strategies to generate, 
manipulate, and use visual images in the process of understanding 
math concepts, whereas in the past, visualizations had been largely a 
product of students' mathematics. 

The supercalculators allowed the successful introduction of two 
modern topics which often could not be covered in previous offerings 
into the linear course: the interpretation of Gaussian elimination as an 
LU-factorization and its application to linear systems with multiple 
right-hand sides, and the interpretation of the Gram-Schmidt process 
as a QR-factorization and its application to least squares problems. In 
these cases, students were able to grasp and handle complex con- 
temporary math concepts, something that had not occurred in the 
past. However, data from a carefully controlled experiment to this 
effect are not available. A new level of testing on new topics has been 
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opened up through using graphics calculators that is not possible in a 
more computationally restricted environment. 

In interviews and questionnaires, the majority of students across all 
experimental courses reported that the graphics calculators helped 
them understand because they could visualize the mathematical 
relations by use of graphs. Almost all students claimed that the 
graphics calculators were useful in solving problems, often because of 
their ability to store programs to be recalled later for complex 
solutions. Students also said that they sensed the opportunity that 
graphics calculators gave them to explore and investigate beyond what 
they could achieve without them. Class attendance in the calculator- 
enhanced classes was dramatically higher than in other sections. 

Clemson faculty have felt the impact of these reforms as well: the 
first seminar on teaching within the math department in over 20 years 
grew out of pedagogical issues surrounding these new courses. Cur- 
rently, instructors meet in three seminars every week about each of 
three courses. Growing acceptance of the experiment from Clemson 
administrators and faculty is reflected in the number of calculator 
sections over time: 6 in 1988-1990, 33 in 1990-1991, 52 in 1991-1992, 
and 56 in 1992-1993. 

This project was selected in 1990 by the Mathematical Association 
of America as one of the ten exemplary mainstream calculus reform 
projects in the nation, and was specifically featured in its publication, 
Priming the Calculus Pump: Innovations and Resources . 

Project Continuation 

By the end of the 1991-1992 academic year, 3,500 students and 26 
math instructors had studied calculus, differential equations, linear 
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algebra, and statistics in calculator-enhanced classes. The project is 
firmly institutionalized within the Clemson curriculum, with plans to 



substantially expand the program through National Science 
Foundation funding. A recent award to Clemson and the Georgia 
Institute of Technology from NSF will extend and adapt the methods 
and materials across the two campuses. 

Currently, the 52 calculator-enhanced class sections represent about 
one-half of all offerings in science and engineering for undergraduates. 
By the fall of 1994, the project plans to integrate the calculator tech- 
nology into two new areas: 31 sections and 1,000 students in college 
algebra and precalculus courses, and 55 sections and 2,100 students in 
business calculus courses. 

Project faculty have given approximately 135 presentations, many 
of them by invitation and including the nation's foremost math and 
technology conferences. A growing list of colleges and universities, 
including Duke University, the University of South Carolina, Lock 
Haven University, and several technical colleges in South Carolina and 
community colleges in New Jersey have adopted the project's methods 
and materials. 

Based on this path-breaking work in improving calculus instruction, 
FIPSE has recently funded another three-year project at Clemson, this 
time for revitalizing nonstandard calculus, the calculus taken by stu- 
dents in economics, management, business and the social sciences. 

In 1992 the project faculty published a series of five course supple- 
ment books through Harcourt Brace Jovanovich and Saunders College 
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Publishing on the pedagogical use of high-level graphics calculators: 



Iris Brann Fetta, Calculator Enhancement for Introductory 
Statistics, 1992. 

D.R. LaTorre, Calculator Enhancement for Linear Algebra . 1992. 

T. G. Proctor, Calculator Enhancement for a Course in 
Differential Equations . 1992. 

J. H. Nicholson, Calculator Enhancement for Single Variable 
Calculus . 1992. 

J. A. Reneke, Calculator Enhancement for a Course in Multi- 
Variable Calculus . 1992. 

Twenty-seven articles have been either published or are forth- 
coming in an array of math and other professional journals on the 
local and national impact of the experiment. Several manuals on the 
use of graphics calculators in algebra, precalculus, and statistics will 
also be published this year by Harcourt Brace. 

For more information about the project and its publications, 
contact: 

Donald R. LaTorre 

Department of Mathematical Sciences 

Clemson University 

Clemson, SC 29634-1907 

803-656-3437 FAX: 803-656-5230 

e-mail: latorrd @clemson.clemson.edu 
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DENISON UNIVERSITY 



Improving the Economics Curriculum with Laboratory 

Experiments 

Purpose 

The work that economists do has changed radically in recent years. 
Whereas in the past they concerned themselves principally with 
theory, refinements in methodology and accessibility of statistical and 
forecasting software have imparted a markedly empirical character to 
the profession. The teaching of economics, however, has not kept pace 
with these changes. In the classroom, the approach is still over- 
whelmingly theoretical, delivered mostly through lectures, with an 
occasional session devoted to discussion. 

The Economics Department at Denison concluded that the best way 
to help students learn to think like economists was to give them 
experience with the problems and situations that economists confront 
every day. In the course of designing these experiences, the faculty 
established the first laboratory economics curriculum in the nation. 

The entire Department participated in the elaboration of this 
curriculum, which was developed and phased in over a period of two 
and a half years. The Department added a laboratory component to 
eleven of its offerings, three of which are core courses, thus ensuring 
that majors graduate with a minimum of four laboratory courses. 



109 



Innovati ve Features 



In the laboratories students systematically gather and use empirical 
information to discover economic principles at work, to test the use- 
fulness and validity of the theory developed in the lectures and dis- 
cussions, and to make informed and reasoned decisions about economic 
activity and policy. Denison's economics majors emerge from their 
studies possessing empirical exposure across the curriculum in addition 
to a theoretical analysis core. 

Evaluation 

The Department designated one of its members as evaluation 
director for the project. With the assistance of outside consultants, 
faculty designed an evaluation plan that included traditional eval- 
uations of cognitive results and alternative approaches to measuring 
affective changes. 

The performance on the Test of Understanding of College 
Economics (TUCE) on the part of 73 macroeconomics students who 
had been taught in the traditional lecture format in 1980 was com- 
pared to the performance on the same test by 80 students who took 
macroeconomics with the laboratory component in 1990. The TUCE 
was administered at the beginning and at the end of the courses. 
Both generations of students had basically equivalent backgrounds, 
and were taught by the same instructor. 

Students also took two sets of value and attitude inventories, one 
normed on high school graduates and one on professional economists. 
In addition, the Department devised its own survey of the ways in 
which students' reading habits, attention to news, and course and 
career choices change in the three years of study for the nuyor. 
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M qjor Ins i ghts and Lessons Learned 



Preliminary results of the comparison of performance on the TUCE 
suggest that the laboratory has a significant effect on sr.udent learning. 
Students who took macroeconomics with the laboratory component 
showed a positive change of 10.7% between pre and post-test scores. 
The scores of students who did not have the benefit of the laboratory 
showed an increase of only 4.2%, 

In the affective realm, equally marked changes have become 
evident. The professional attitudes survey indicates that students who 
took laboratory courses did indeed begin to think more like 
professional economists. In the classroom, faculty find that these 
students write more and better papers, and undertake independent 
studies and honors projects. They use empirical evidence more 
effectively and are more enthusiastic about their major than their 
predecessors. 

Perhaps due to the more personal atmosphere brought about by 
laboratory interactions, the number of women economics majors has 
increased from about 35% to a little over 50%. Likewise, the number 
of minority economics majors has increased from four or five a year to 
ten or fifteen. Furthermore, the laboratory has become a gathering 
place for all students, who are cementing their new-found sense of 
community by resurrecting defunct clubs and organizations. 

The project has brought about a revitalization not only of every 
course in the curriculum, but of faculty as well, A new style of 
collegiality has emerged as young instructors assist senior professors 
with computer use. The laboratory format has also fostered a more 
personal relationship between faculty and students. 
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Unanticip ated problems 



It is tempting for faculty engaged in laboratory design to produce 
simply another form of computer assisted instruction, with closed-end 
exercises that are easy to grade. As Denison faculty gained more 
experience, they became better able to use generic software to create 
"discovery" laboratories or simulations in which, for example, students 
reason from the findings of data analysis to develop new (to them) 
theoretical approaches to economic problems. It is in these labs that 
students learn that received theory does not always explain observed 
phenomena. 

Project Continuation 

The lectureAaboratory curriculum has been formally put in place 
by the Denison governance system. Only the introductory economics 
course has been omitted from the lectureAaboratory format, due to 
staffing limitations and because this course satisfies a general edu- 
cation requirement. Two additional laboratory courses have been 
formulated, and a new social science freshman course, team-taught 
by economics, sociology and anthropology faculty, uses the empirical 
approach first adopted by the Economics Department. 

Recognition 

Denison* s economics curriculum was highlighted by the Association 
of American Colleges at its 1990 national meeting. The Economics 
Task Force Report included in the AAC's Liberal Learning And The 
Arts And Sciences Mqjor recognizes Denison's program and one other- 
out of a survey of 400-for excellence in undergraduate instruction. 
This report has been published in The American Economic Review and 
in the Journal of Economic Education . 
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Many institutions have shown interest in adapting the lecture/ 
laboratory Economics curriculum. Based on the curriculum's success, 
one of the project directors was invited to testify before a Senate 
Subcommittee on the effectiveness of FIPSE grants. 

Available Information 

Denison University will make available copies of the grant proposal, 
of the various reports from the American Economics Association and 
the Association of American Colleges Economics Task Force, and of 
several publications of preliminary results. 

Requests should be addressed to: 

Robin Bartlett 
Department of Economics 
Denison University 
Granville, OH 43023 
614-587-6245 
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DICKINSON COLLEGE and TUFTS UNIVERSITY 

Workshop Physics 
and Tools for Scientific Thinking 

Purpose 

The avorage introductory physics textbook is 1,000 pages iong The 
average physics student emerges from his first course in a state of 
cognitive overload, retaining at best a few formulas and definitions, 
and clinging to some of the very misconceptions about physical 
phenomena that the course was des -;ned to dispel. 

The Workshop Physics project at Dickinson College and the Tools 
for Scientific Thinking project at Tufts University were conceived to 
address the problems cf teaching and learning introductory physics. 
Project fa" determined that these courses should prepare students 
for further study of physics and other sciences and should enable them 
to become proficient in the use of computers and other research tools. 
Most importantly, introductory physics courses should whet students' 
appetites to learn more science. 

The projects, which originally were funded separately by FIPSE, 
merged in their third year* 

Innovative Features 

The Workshop Physics project resulted in the creation of two two- 
course sequences, one in calculus-based introductory physics and one 
in algebra-based physics. The principal dilemma confronting 
introductory physics instructors is choosing what to teach and how to 
teach it: relativity, quantum theory and chaos vs. Newton's Laws and 
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classical thermodynamics; new pedagogies based on cognitive theory 
vs. traditional instruction; and the digital computer vs. the electronic 
calculator. Accordingly, one of the faculty's most significant decisions 
was to reduce the amount of information delivered to the students in 
favor of enabling them to acquire transferable skills of scientific 



The choice of which topics to retain in the syllabus was based on 
the extent to which these lent themselves to direct observation, and to 
the broad applicability of the mathematical and reasoning skills 
required to analyze these observations. Topics such as relativity and 
quantum mechanics, which require levels of abstract reasoning beyond 
the abilities of beginning students, were eliminated. 

Since all pedagogical decisions were oriented towards the goal of 
teaching students to think scientifically, lectures and demonstrations, 
whose value lies in transmitting information rather than helping 
students to reason like scientists, were abandoned in favor of direct 
inquiry and discussion. 

The microcomputer is an essential tool in this approach. Not only 
can students work scientific operations quickly, but a microcomputer 
connected to a sensor (such as an ultrasonic motion detector, photo- 
gates, temperature sensors or geiger tubes) via an electronic interface 
can perform instantaneous calculations and produce graphs. The use 
of a microcomputer-based laboratory (MBL) to produce a time trace of 
the position of a student's body, for example, can help that student 
grasp intuitively how a graph represents the history of change in a 
given parameter. 



inquiry. 
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Evaluation 



Using essays, homework and laboratory completion rates, per- 
formance on various tests, and interviews, project faculty measured 
student attitudes, conceptual development, problem-solving abilities, 
and computer and laboratory skills. The most fruitful procedures 
included a comparison of student ratings on the all-college rating 
system which has been in use for ten years at Dickinson; a survey of 
course management practices and student attitudes administered to 
over 3,000 students at 22 institutions; and pre- and post-tests of 
introductory physics students over a period of four years. Although, 
given the volume of evaluation data, analysis is not yet complete, the 
project has yielded significant gains among the more than 250 
Dickinson students who have completed Workshop Physics courses 
since 1987. 

Attitudes towards the study of physics have improved, as shown by 
the numerical ratings of calculus-based Workshop Physics students on 
the standard all-college evaluation forms. About two-thirds of the 
students in these courses express a strong preference for this method 
over the lecture approach, and enrollments in advanced courses have 
increased by 15-30%. Results for the algebra-based courses serving 
premedical students are less positive-only half of the students in these 
courses prefer the workshop approach. 

Because of the direct experience with phenomena allowed by 
Workshop Physics, a greater percentage of students master concepts 
that are usually considered difficult to teach because they involve 
classic misconceptions. For example, Dickinson students show 
significant gains in graph interpretation skills needed for the study of 
kinematics after learning to interpret velocity graphs by producing 
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graphs in real time on a computer screen with the aid of an MBL 
motion detector system. On the other hand, Tufts University students 
who listened to traditional kinematics lectures did not show a signifi- 
cant reduction in error rates. 

Although the conceptual gains of Workshop Physics students are 
greater than those achieved by students taking conventional physics 
courses in many topic areas, the gains are not universal, and in certain 
areas Workshop Physics students perform no better that their tra- 
ditional peers. 

Student performance in upper-level physics courses and in solving 
traditional textbook problems is as good as or better than that of 
students in the traditional curriculum. Moreover, Workshop Physics 
students demonstrate a comparatively greater degree of comfort 
working with computers and other laboratory equipment. 

Regardless of format, female students react less positively to the 
study of physics than males. Nevertheless, women who enroll in 
Workshop Physics make proportionately greater gains than males in 
their appreciation of the use of computers. Despite women's supposed 
dislike for hands-on work, female students continue to comprise 30% 
or more of the physics majors at Dickinson. 

Some students perceive Workshop Physics as taking more time out 
of class than the traditional approach, although surveys demonstrate 
that this is not in fact the case. A small number of students dislike 
the hands-on approach, and would much prefer a textbook and lecture- 
oriented pedagogy. 
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Project Impact 



Despite the adjustments in schedule and teaching style required 
initially (the courses are taught in three two-hour sessions per week 
rather than the traditional three hour-long lectures and single two- or 
three-hour laboratory), Workshop Physics is now regularly offered at 
Dickinson. The amount of national attention the courses have 
attracted has helped to make them popular with students, faculty and 
administrators. Nevertheless, instituting academic change is a 
laborious and time-consuming process. 

The assessment of a project such as this one can become 
problematic if it is not tied carefully to evaluation of students for 
grading purposes. Improperly planned assessment can yield mountains 
of unanalyzed data, and can rob faculty of valuable time. 

Workshop Physics has so far resulted in impressive gains for 
students, yet much remains to be learned about how individuals learn 
different scientific topics. Workshop Physics is certain to continue to 
generate subjects for investigation and reflection for years to come. 

Recognition 

The project has received a number of national awards, such as the 
Merck Innovation Award in Undergraduate Science Education and an 
EDUCOM/NCRIPTAL Award for Best Software in Physics. The 
Director of Workshop Physics, Priscilla W. Laws, was awarded a 
Distinguished Service Citation from the American Association of 
Physics Teachers. The project has garnered close to three million 
dollars in additional grant funds from FIPSE, NSF and IBM. 
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Project Continuation 



The developers of Workshop Physics are collaborating formally with 
faculty at the University of Oregon, Ohio State University and Rutgers 
University in the adaptation and testing of Workshop Physics and 
Tools for Scientific Thinking materials. Over 50 other institutions use 
these materials to various degrees and have adapted the project's 
curriculum to their own needs. 

In addifc m, the project has attracted the attention of faculty from 
chemistry, biology and mathematics at several colleges and 
universities. Their joint interdisciplinary efforts include designing 
software, devising uses for the laboratory technology, and testing. At 
Dickinson, a new Workshop Mathematics program, recently funded by 
FIPSE, is investigating ways of adapting Workshop Physics to 
mathematics instruction. 

Available Information 

Copies of articles about Workshop Physics, and information 
regarding workshops for faculty interested in learning this approach 
may be obtained from: 

Virginia Trumbauer 

Department of Physics and Astronomy 

Dickinson College, Box 1773 

Carlisle, PA 17013-2896 

717-245-1845 

Internet: TRUMBAUE@DICKINSON.EDU 



Copies of the Workshop Physics Student Activity guides, both 
calculus-based and algebra-based, as well as an apparatus guide are 



available in both printed end electronic formats for Macintosh 
computers. The cost is nominal and purchasers are granted 
permission to modify and/or reproduce materials locally. The 
Microcomputer-Sased Laboratory apparatus, including a serial 
interface, sensors, and Macintosh software developed jointly at Tufts 
University and Dickinson College are also available for purchase. 

Price lists and ordering information are available from: 



Vernier Software 
2920 SW 89th Street 
Portland, OR 97225 
503-297-5317 
Fax: 503-297-1760 
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UNIVERSITY OF MARYLAND, BALTIMORE COUNTY 



The Satellite Development Project in French Language 
and Culture: France-TV Magazine 



Purpose 

The project had four goals: 1) to fill the need in French language 
and area studies for culturally authentic, lively materials received in 
the classroom in a continuing fashion; 2) to create a model of tele- 
vision satellite-assisted instruction that would foster a communication 
approach to the teaching of foreign languages; 3) to produce and 
distribute nationwide and on a monthly schedule supporting instruc- 
tional materials; and 4) to disseminate this high tech instructional 
system in such a way as to make it financially self-supporting. 

Project Activities 

The medium through which these purposes are carried out is 
France-TV Magazine , a cooperative effort of France 2, the leading 
French TV public network; Mediane Films, a small production 
company; and the University of Maryland, Baltimore County (UMBC). 
The 60-minute satellite-transmitted television program consists of 
major news stories of the past month assembled from the network's 
broadcasts, and features on contemporary France and Europe. The 
magazine appears monthly during the academic year. The broadcasts 
are accompanied by a study guide and teaching resources assembled by 
a team of UMBC staff and school teachers. 
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In the second year of the project, marketing and distribution, 
including both print and diskette versions of the instructional support 
materials, were taken over by the Public Broadcasting Service (PBS). 
Soon after, secondary distributors (for example, Sat Link in Missouri 
and station WIZ in Cleveland) started to set up local computer bulletin 
boards (BBS) listing program updates and providing a medium for the 
exchange of pedagogical ideas. 

The TV Magazine and support materials were made available in a 
variety of versions and in a number of different formats. They could, 
for instance, be obtained in libraries and in a home video form. 

Innovative Features 

The project is unusual in its provision of current authentic 
materials supported by at least 65 pages of instructional text per 
program, thus constituting a unique language and culture video 
"textbook." France-TV Magazine has also developed a practical "step- 
by-step" video-based pedagogy specially designed for immediate use in 
the classroom. Widely distributed, the France-TV Magazine concept 
has reached quite a remarkable viewership. Indeed, in 1993, France- 
TV Magazine is in use in well over 200 colleges and universities, 
resulting in an estimated half-million viewers annually. Secondary 
school students in 200 cities in 35 states have access to the program, 
as does the general viewing audience for the PBS affiliates and cables 
carrying the program. The overall potential audience has been 
estimated at over 70 million viewers. 

Evaluation 

Teacher responses and ratings have been gathered systematically 
regarding 1) the situations in which France-TV Magazine has been 
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used, 2) the features of the program that have been selected, and 3) 
the aspects of language learning on which the material has had the 
most influence. 

It was found, for instance, that only short video segments (no more 
than four minutes) should be used in the foreign language classroom. 
Subjects dealing specifically with youth: lifestyles, issues, etc., and the 
French educational system were in demand, and they were therefore 
enhanced in the program. The questionnaires also included the eval- 
uation of technical aspects of the program such as the quality of 
picture and sound reception. Students were not asked to rate the 
materials. However, each innovation in France-TV Magazine's ped- 
agogical approach was carefully tested in the classroom before being 
integrated in the support materials. The acute emphasis on the 
importance of contextualization and preview activities was a result of 
these in situ experiments. 

The continued involvement of PBS and the System's efforts to 
make France-TV Magazine a self-supporting operation are a strong 
endorsement of the quality of the work and its real usefulness to a 
broad audience. 

Project Impact 

The video magazine has had its greatest utilization in classes at the 
intermediate and advanced levels. It provides authentic materials 
which help students understand how the language is used in France 
today in its fully nuanced form, teaching not only the content of the 
culture but reinforcing the idea of language as both a cultural artifact 
and a carrier of culture. Thus, the materials have proven an ex- 
tremely valuable support for principles and strategies of language 
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teaching that are rapidly becoming the norm in the foreign language 
classroom. 

The electronic Bulletin Board Service, originally intended as the 
medium of distribution for the pedagogical materials, has become 
instead a means of conveying information about future programs and 
allowing users to exchange ideas, organize conferences, and trade 
teaching tips. The pedagogical materials are now available as print 
materials and IBM-compatible diskettes. 

The goal of making the service self-supporting was finally reached 
in 1992 when UMBC took over the distribution of videocasettes and 
was able to retain all the income from that source rather than sharing 
it wii a PBS. In order to reduce the cost of the magazine service and 
thus expand the audience, some specially produced cultural material 
had to be dropped. Target segments in all levels were developed as 
user samples. 

Major Insights and Lessons Learned 

The growing use of France-TV Magazine at a full range of colleges 
and universities and at over 1,000 schools reached through 327 PBS 
affiliates nationwide has demonstrated the need for such a program. 
Over the years, however, authentic materials have proven to be most 
valuable only if carefully selected and accompanied by support 
materials. 

In order to make France-TV Magazine self-supporting, a leaner, 
less high tech version than the original project was developed to keep 
down costs and expand the audience. 
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France-TV Magazine is now the official program of the Delegation 
Generale k la Langue Franchise in Paris, Since the production is now 
entirely funded by the Ministere des Affaires etrangeres, the program 
is distributed to all French cultural services, in more than one hun- 
dred countries, and is present on the five continents. The program is 
being aired in both Americas, Japan, Tanzania, the former USSR and 
throughout Europe, Negotiations are still taking place in Paris to 
widen even further France-TV Magazine's audience, 

France-TV Magazine won two red ribbon awards at the 32nd and 
34th annual American Film and Video Festival in San Francisco and 
Philadelphia, 



Available Information 



Information about satellite reception for colleges, universities and 
high schools is available from: 



PBS Adult Learning Satellite Service 
1320 Braddock Place 
Alexandria, VA 22314 
1-800-257-2578 
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Information about pre-recorded videocasettes is available from: 



France-TV Magazine 
AC-IV, Room 146 

University of Maryland, Baltimore County 
Baltimore, MD 21228-5398 
1-800-992-3788 or 
410-455-2963 

General information about the project is available from: 
Claud Du Verlie 

Department of Modern Language and Linguistics 
University of Maryland, Baltimore County 
5401 Wilkens Avenue 
Baltimore, MD 21228 
410-455-2130 
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NEW MEXICO STATE UNIVERSITY 
Overview, Case Study Physics 



Purpose 

Overview, Case Study Physics, is a "spiral" form of instruction in 
introductory physics based on the last two decades of cognitive and 
physics education research. The course is divided into a small number 
of conceptual blocks. Each block begins with an overview in which 
students construct or acquire a qualitative understanding of the basic 
physical processes under study using diagrams and graphs. In the 
second part of the block, students learn the mathematical representa- 
tions of the same concepts, using words, sketches, diagrams, graphs 
and basic equations. They then learn to shift back and forth between 
the various forms of representation. Following this "exposition" 
section, students solve case study problems involving the integrated 
use of knowledge introduced earlier in the course. The overview- 
exposition-case study process is then repeated for the next block of 
material. Throughout, students spend the majority of the class time 
working together in small groups. 

Innovat ive Features 

Instruction systematically incorporates pedagogical techniques 
whose validity has been consistently confirmed: active learning; 
collaborative learning; presentation of subject matter in a variety of 
forms, both qualitative and quantitative; presentation of ideas in 
successively more sophisticated forms at increasing levels of 
complexity; and careful sequencing of ideas. 
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The course approach is presented in a study guidv. and a set of 300 
Active Learning Problem Sheets (ALPS). The former supplements the 
standard text, and contains materials for the overviews, activities to 
support the multiple representation problem solving, and case study 
problems. 

The ALPS Kit helps students organize their thinking, reason 
qualitatively about physical problems and change the representation of 
a problem. It can be used for collaborative learning during the class 
period, for homework assignments or for quizzes. Some of the case 
study problems require students to divide problems into parts and give 
definition to poorly defined problems, thus giving students experience 
at the introductory level in solving unstructured problems. 

Evaluation 

Strikingly superior student performance using the overview, case 
study (OCS) approach as opposed to standard lecture approaches has 
been demonstrated in five institutions of different types. In courses 
using standard strategies, student gains in understanding, as measured 
by pre- and post-tests, are small and independent of instructor style 
and ratings. Gains registered by students in OCS courses, on the 
other hand, are not only several times greater, but 75-80% of those 
who begin the course finish it, as opposed to about 60% in the 
conventional courses. 

Other studies of the comparative results of the OCS approach have 
shown that: 

1. OCS students do better than traditional students in absolute 
end-of-course test scores. 
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2. Students taught with the OCS strategies retain their learning 
better, as measured by a diagnostic test given at the beginning of 
the second semester course two semesters after the first course was 
completed. 

3. Students in an interface course offered for poorly prepared 
students and using the OCS approach performed better at the end 
of the course than students in the regular course. (The problems 
on the comparison test were written by faculty associated with 
neicher course.) 

4. Students in OCS sections do markedly better than traditional 
students on a final examination made up of problems written by 
instructors in traditionally taught sections of the same course. 

Ratings of student satisfaction with the course are extremely high-- 
9.2-9.8 on a 10-point scale. 

Project Impact 

The materials have been used successfully at a variety of 
institutions, from state colleges and universities to technical colleges 
and two-year colleges, but seem to have found particularly wide 
acceptance at the latter. The project director has conducted work- 
shops and colloquia at NSF-sponsored workshops for community 
college teachers, at meetings of the American Association of Physics 
Teachers, at ten U.S. universities and in South Africa. Estimates are 
that 20 or more institutions are currently using the materials. 
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Mqjor Insights and Lessons Learned 



The oversight and case study method of instruction clearly 
demonstrates the value of paying attention to specifically relevant 
research on student learning. By designing a course that responds 
systematically to the clues coming from research findings, the project 
has yielded an instructional approach that produces demonstrably 
superior student outcomes and high student satisfaction. 

Recognition 

The OCS approach was included in the Introductory University 
Physics Project of the American Institute of Physics, which dis- 
seminates outstanding new instructional developments to the disci- 
pline. To join the Physics Project, instructional models had to be 
rated by a panel of physics professors in Lhe top ten of those present- 
ed. The OCS model received the highest ratings of all. 

Project Continuation 

Use of the OCS approach continues to grow. Several publishers are 
considering production of a text based on the model. The ALPS Kit is 
now being used in the lecture and recitation parts of the introductory 
physics course for engineers at Ohio State University. The project 
director recently received an NSF grant to develop a series of labora- 
tories and lecture experiments building on physics education research. 
He will integrate these new lab activities into future FIPSE labs at 
Ohio State University. 
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Available Information 



Information about the course and sample materials may be 
obtained from: 

Alan Van Heuvelen 
Department of Physics 
The Ohio State University 
174 West 18th Avenue 
Columbus, OH 43210-1106 
614-292-5713 
FAX: 614-292-7557 
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NORTHWESTERN UNIVERSITY 



Redesigning the Content and Sequence of Instruction 
in Music Theory 

Purpose 

To the faculty of the Northwestern University School of Music, the 
way music theory is usually taught-note by note and chord by chord- 
seemed to run counter to the way in which music is apprehended and 
appreciated by seasoned musicians, as a coherent whole rather then as 
the sum of disparate fragments. Simply put, where traditional theory 
instruction proceeds from the bottom up, real musicians approach 
compositions from the top down. 

The faculty envisioned the process of musical comprehension and 
performance as a problem-solving task that includes aural, visual, 
kinesthetic, affective and chronological components. Accordingly, they 
set about developing a multi-media, real-time instructional approach 
that would allow the students to perceive these elements as sn inte- 
grated whole. 

The project focused on the freshman core curriculum, which 
consists of courses in musicianship and aural skills. Faculty identified 
four strategies that would result in the kind of musically sophisticated 
grasp of compositions that they wanted students to achieve. Each of 
the four strategies-memorization of compositions, modeling of musical 
structures through creating programs which "compose" these struc- 
tures, analytic reading of musical scores, and analytic listening-became 
the province of a particular faculty member. Each strategy required 
in-class paper and pencil work on the part of the students, and all 
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but the first one involved use of Macintosh software developed by the 
faculty. 

Two of these software systems, the ScoreScan/Imager system for 
teaching analytic reading scores, which includes a library of 200 scores 
for practice, and the EarWorks system for designing and delivering 
analytic listening lessons with commercial audio CDs, are now in 
regular use at Northwestern. The School of Music anticipates pub- 
lishing the analytic listening and score analysis materials as textbooks 
and as computer-assisted instruction software for wider distribution. 

As initially envisaged, the freshman curriculum was to revolve 
around a few core compositions which the students would visit time 
and again from different points of view. The computer would allow 
them to manipulate the music in a variety of ways, including revoicing 
and reorchestrating, and to compose new works based on the models 
studied. 

Evaluation 

Each aspect of the project was evaluated by the faculty member in 
charge of its development, and as a result, some changes were made to 
the original curriculum. Memorization was dropped after one term, 
without the benefit of formal appraisal, since faculty and students 
agreed that the musical gains were not worth the laboriousness of the 
approach. The modeling of musical structures through programming 
was also discontinued, since although students became adept at 
writing about their discipline, the musical quality of their projects was 
relatively low, and they spent a disproportionate amount of time work- 
ing out programming difficulties. 
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In the analytic score reading component, students were given a pre- 
test in which they attained a mean of 57.3. The mean for quizzes 
administered during the course was 90.0. Even though this constitutes 
a comparison of different instruments, faculty believe that it confirms 
their own sense of students' improved abilities in reading scores. 

Progress in analytic listening consists in moving from an initial 
perception of a musical work as a succession of events to a more 
synthetic apprehension of structural relationships between non- 
adjacent points in the work. In this component of the project, stu- 
dents' responses in a listening exercise at the beginning of the year 
were compared to their responses in a similar task two quarters later. 
In the initial exercise students commented mostly on descriptive 
features of the composition, such as a timbre and rhythm. In the later 
exercise, however, comments on the sonic and kinesthetic properties of 
the work decreased, and remarks on the formal qualities of the compo- 
sition, such as harmony and melody, increased. This is all the more 
significant in that the exercise did not explicitly call for comments on 
structural features. 

Analysis of qualitative data-student performance protocol and 
transcript files-has not been completed, but general observations tend 
to confirm the developments outlined in the listening exercises. 

M qjor Insights and Lessons Learned 

To the faculty who devised it, this project demonstrates the 
soundness of the "top-down" approach and the usefulness of computers 
in aiding students to progress towards a higher level of musicianship. 
Whereas initially the faculty had expected that a comprehensive 
curriculum centered on the computer would result, experience made it 
obvious that it was wiser and more realistic, given time and political 
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constraints, to reserve the computer for very specific, high yield 
applications. 

Recognition 

The project, which has prompted a score of papers and 
presentations, has attracted attention from a number of institutions, 
including Yale University, the Eastman School of Music, the Univer- 
sity of Michigan, Indiana University, the University of Illinois and 
Arizona State University. The success of the software designed in 
conjunction with the project has enabled the School of Music to obtain 
funds for its Macintosh laboratory from the Kemper and Wurlitzer 
Foundations as well as other private and corporate sources. 

Available Inform ation 

Syllabi and course-related materials as well as copies of papers and 
presentations may be obtained from: 

Richard Ashley 

School of Music 

Northwestern University 

711 Elgin Road 

Evanston, IL 60208-1200 

708-491-5431 

email ric@music.nwu.edu 
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UNIVERSITY OF OREGON 



Biology Laboratory Construction Kit 
With Intelligent Tutor 

Purpose 

Previous computer simulations used for instruction in introductory 
biology have employed models of a single fixed biologic system, for 
example, simulations in physiology for medical students. This focus on 
particular systems did not permit introductory-level study of cardio- 
respiratory systems across species, phyla or kingdoms. 

The purpose of this project was to overcome these limitations by 
developing a computer toolkit and intelligent tutor that would place 
students in a flexible computer environment for learning basic con- 
cepts of a variety of circulatory systems. Students generate hypo- 
theses, design experiments, test realistic simulations and learn how 
their constructions meet or fail to meet design objectives. The 
program invites investigative thinking, and particularly develops the 
ability to draw conclusions and unravel causality. 

Innovative Features 

The project attempted to connect the best of the constructivist 
approach to the use of computers-the cardiovascular toolkit-with the 
best of the tutorial-the intelligent tutor. The toolkit (programmed in 
Allegro Common Lisp on Macintosh computers) allows students to 
construct a cardiovascular system graphically by hooking together 
appropriate parts. The tutor was designed to guide and critique 
student work. Students start by selecting cardiovascular components 
(hearts, vessels, capillaries) as graphic objects from a menu of icons. 
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Gauges capable of measuring blood pressure, flow, volume and oxygen 
can be attached to the components to visualize the system's status. 
Components and gauges can be moved around and hooked together to 
build new structures. Parameters for each component, such as 
resistance to flow or pumping rate, can be easily modified to create 
changes in cardiovascular behavior. 

Thus, by selecting and placing graphic objects on the computer 
screen, the components can be connected in a number of topological 
ways. The user can then run the system by clicking on a control 
panel, and can, if he or she chooses, accelerate or animate heart beats, 
blood flow and one-way valve operation. Graphing windows associated 
with gauges immediately display factors such as pressure, flow, and 
oxygen content. 

The curriculum built around the tutor is designed for two-hour lab 
periods, supported by lecture and pre- and post-lab discussion sessions. 
It first acquaints students with model behavior and instructs them on 
how to pose problems and design experiments. In later laboratories, 
students devise their own hypotheses, design their own experiments, 
and practice drawing conclusions based on results. The software tutor 
helps instructors coach or critique students as they are making 
constructions. 

Evaluation 

« 

Student and instructor curricular materials were tested in small 
groups, using four experiments on the cardiovascular system. A 
second lab evaluation tested students* ability to generate hypotheses 
and draw logical conclusions. A third tested students' skill at maxi- 
mizing the performance of a construction for producing, transporting, 
and consuming oxygen. 

140 

445 



Project staff used videotapes of pairs of students using the lab 
instructions and simulations to assess the adequacy of the software 
interface, the instructional design, and prior student knowledge, 
abilities and misconceptions. Extensive testing during lab experiments 
identified key problems in both the students' biological understandings 
and in the human-machine interface. The students' misconceptions 
turned out lo be deeply held and related to basic principles of physics. 
Fortunately, the videotapes showed that the software helped eliminate 
the misconceptions, as students discovered for themselves the flaws in 
their logic and in their assumptions. Both the videotapes and class- 
room testing showed that students and faculty using the software 
found it instructive and easy to use. In fact, most students learned to 
use the program without the manual. 

The Construction Kit and curriculum were distributed to 18 
additional sites for field testing (universities, colleges, community 
colleges and schools) as part of the testing of BioQuest software 
(Quality Undergraduate Educational Simulations and Tools in Biology, 
Beloit College). Although BioQuest requested evaluations of the 
software from all of its test sites, especially on student use and 
learning, only a few sites responded. Most comments from field sites 
were related to problems in the software (especially regarding com- 
patibility problems with various versions of the Macintosh system). 

Project Impact 

The Construction Kit and curriculum are being used by over 500 
students each year in at least four different biology courses at the 
University of Oregon. The software has become part of a larger cur- 
ricular project (funded by FIPSE and the National Science Founda- 
tion) designed to encourage investigative learning in biology courses 
for non-majors. 
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Unanticipated Prnhlpmg 



Developing causal explanations for the behavior of arbitrary 
constructions proved to be more complex for the faculty than 
anticipated. Also, students' false expectations confounded inter- 
pretation of test data. Definitional problems arose out of the testing 
of "normal" versus "abnormal" physical properties and of absolute 
versus relative measurement of these properties. 

Although significant progress was made in developing the 
intelligent tutor, it proved unrealistic to incorporate the tutoring 
software into the final software product within the time-frame of the 
project. 

Major In «iflht« an d Lessons Learned 

Faculty often had difficulty pinpointing the nature of student 
problems simply from observing students using the software. When a 
student felt the simulation had behaved unexpectedly, for example, 
there were at least four possible explanations: 1) the student did not 
under- stand how the simulated system represented the real systems 
being simulated (a problem with the interface); 2) the simulated 
system did not accurately represent the real system (a problem with 
the Simula- tion model itself); 3) the student had misconceptions about 
the real system, and/or 4) the student was misinterpreting the results. 

The first two explanations complicate the use of simulations in 
education and underscore the need for thorough testing of both the 
user interface and the simulation model. If a simulation is to erase 
student misconceptions about real biological systems, then both 
students and instructors must have confidence that what they are 
seeing in the simulation accurately reflects what happens in real life. 
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Otherwise, it is not possible to know whether to focus on the student 
or on the software when something unexpected happens. These prob- 
lems were especially acute for this software because of its ability to 
design systems of arbitrary complexity, challenging even expert faculty 
to know what to expect. 

Generally, designing and building effective educational software is 
very difficult, time-consuming and expensive. Testing the pedagogical 
goals of the software while it is being developed and while it is in use 
is essential, but very little software is subjected to this kind of review. 

Project Continuation 

The software is used in several University of Oregon biology 
courses. Testing at the 18 sites continues; therefore, neither the 
software nor the curricular materials are quite ready for final pub- 
lication. When FIPSE support ended, the University, Apple Computer, 
and BioQUEST funded the last stages of the project. The software 
will be published on CD-ROM with other BioQUEST materials in 1993 
by the University of Maryland Press. 

One of the project directors, the Chair of the Biology Department, 
was awarded a second FIPSE grant of $257,000 to support workshop 
biology for non-majors. This project, which promotes scientific literacy 
through investigative labs and issue-oriented activities, was influenced 
by the student-directed and open-ended investigations of the Construc- 
tion Kit. Recently, the toolkit has been modified to be more useful in 
a workshop format. 
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Available Information 

Beta versions of the software (the Cardiovascular Construction Kit) 
and the lab and users' manuals are available from the project 
directors: 



The software runs on Macintosh computers. It requires a hard 
disk and at least four megabytes of memory. Please send two empty 
800-kilobyte or high-density diskettes. In return, the project directors 
ask for help in evaluating the software and curricular materials at 
additional campus sites. 

The Intelligent Tutor is also available, as is a 17-minute videotape 
of the toolkit in use. 

The software and manuals are also being distributed as part of the 
BioQUEST project, BioQuest simulations are published on CD-ROM 
by the University of Maryland Press. For information about 
BioQUEST, write to: 



Daniel Udovic 
Department of Biology 
University of Oregon-POB 3158 
Eugene, OR 97403 
503-346-6051 



Nils Peterson 

From the Heart Software 

Eugene, OR 97403 



Sarah A. Douglas 
Department of Computer and 
Information Science 
University of Oregon 
Eugene, OR 97403 
503-346-3974 
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John Jungck 
Project BioQUEST 
Beloit College 
Beloit, WI 
608-363-2743 

Several articles based on the project include: 

Douglas, S., Peterson, N.S., and Udovic, D. "The Cardiovascular 
Construction Kit" in BioQUEST: Quality Undergraduate Educational 
Simulations and Tools . University of Maryland Press. 1993. 

Douglas, S A and Zheng-Yang Liu. "Generating Causal Explanation 
from a Cardiovascular Simulation" in Proceedings of the International 
Joint Conference on Artificial Intelligence , Detroit, ML August 1989. 

Downing, Keith Linn. 1990. The Use of Teleology in the Qualitative 
Evaluation and Explanation of Circulatory Systems. Ph.D. Disserta- 
tion. University of Oregon. (Available through University Microfilms.) 

Novak, David. "A Laboratory Curriculum for the Cardiovascular 
Toolkit". Thesis submitted in partial completion of the Master's 
Degree, University of Oregon, Summer 1989. 
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UNIVERSITY OF RHODE ISLAND 



Chem-Tutor: An Expert System For Teaching Freshman Chemistry 
Purpose 

After a freshman chemistry course, most students are able to use 
the information they have acquired to solve simple, carefully-struc- 
tured problems. When confronted with an exercise that requires them 
to use more than one principle, or that does not explicitly state 
necessary relationships, however, students are likely to falter and give 
up without even attempting to think their way through the problem. 

Research indicates that success in solving complex problems is 
highly correlated with hours of practice-thousands of hours of 
practice-and with appropriate feedback. Yet often introductory sci- 
ence classes number in the hundreds of students, and teaching assis- 
tants typically teach three laboratory sections per semester. Thus, 
neither faculty nor teaching assistants are able to provide students the 
amount of practice and feedback they require to learn to solve complex 
problems. This project addressed that situation through the design, 
writing and testing of software for an intelligent computer-assisted 
instructional system, Chem-Tutor, that enables first-year chemistry 
students to improve their ability to solve complex problems. 

Innovative Features 

Chem-Tutor integrates an intelligent tutoring system into a tra- 
ditional academic setting. It improves the complex problem solving 
ability of the student by articulating its own reasoning skills and 
explanations, coaching the students problem solving behavior directiy, 
and diagnosing his or her errors. 
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Chem-Tutor consists of a domain expert, which contains a 
knowledge base that serves as a ncdel for problem solving concepts 
and procedures, and a problem manager, which controls a library of 
over 1,200 problem templates It also includes a tutor expert, which 
comprises the rules and procedures for 'infraction with the student, 
and the user environment -on-line tools such as a full-screen Periodic 
Table of the Elements. The p rogram runs on a DEC VAX computer, 
contains over 50,000 lines of Comiiion Lisp code, and is designed to be 
used at least one hour a week over an entire semester. 

In the course of the project, a ^separate, complementary program, 
Chem-Tutor Instructor Expert, was also developed. This tool allows 
the instructor to reorganize, add, or delete areas of Chem-Tutor, to 
generate reports on the status of individual students or of the entire 
class, and to intervene when Chem-Tutor indicates that a student is in 
difficulty. 

Evaluation 

Approximately 600 students in The University of Rhode Island's 
beginning chemistry classes rated their satisfaction with Chem-Tutor 
6.8 on a scale from 0 (poor) to 10 (excellent). There was a large 
amount of variation from the average, and this variation correlates 
with the quality of the student's relationship with the teaching 
assistant. 

After using Chem-Tutor, student performance on the final exam in 
the fall 1988 semester was compared to the final exam grade 
distribution for the fall of 1986, before Chem-Tutor was introduced. A 
check of the SAT scores for these two classes separated by three years 
shows no significant difference, and the sections were taught by the 
same instructor at the same time. One can therefore assume that the 



148 

152 



1986 and 1988 students were equivalent in background, ability, and 
SAT scores. The mean exam score for 1986 was 52.6%, and the mean 
exam score for 1988 was 56.6%. This is almost a 10% change in the 
average grade. Using a t-test> the comparison revealed a significant 
difference between students who had used Chem-Tutor and those stu- 
dents who had not (t(683)=2.79, p<0.01). 

Surprisingly, despite the availability of Chem-Tutor the percentage 
of completed homework assignments per student was as low as 63.5%. 
Nevertheless, final grades did correlate positively with the percentage 
of homework completed-that is, students who completed more home- 
work received higher final grades. 

Project Impact 

Chem-Tutor has been used by over 4,000 URI students to solve 
some half million chemistry problems. The required use of Chem- 
Tutor has increased the time that students spend solving problems, 
and the time spent tutoring on the part of instructors has decreased 
proportionately. The project has been institutionalized at the 
University of Rhode Island since 1989. 

Several other institutions, including the University of Wisconsin, 
SUNY Albany, the University of New Hampshire, and Milan Poly- 
technic University have purchased Chem-Tutor and are adapting it to 
their own needs. 

Unanticipated Pr oblems 

In the course of the project, efforts to develop a natural language 
user interface had to be abandoned due to the complexity of natural 
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language processing. The designers of Chem-Tutor are attempting to 
overcome the system's limited ability to use artificial intelligence. 

Because Chem-Tutor is text-based and keyboard-driven, its 
usefulness in certain areas requiring use of graphics is insufficient. 
Future systems will operate in a Graphical User Interface environ- 
ment. 

M ajor In sights an d Lessons Learned 

As a result of experience gained in the development and use of 
Chem-Tutor, it has become apparent that any successful tutoring 
system must take into account the four elements that make up an 
integrated learning environment-the discipline, the available 
technological resources, the instructor, and the learner— and their 
interactions with each other. 

Project Continuation 

An analogous program, Math-Tutor, was created after Chem-Tutor, 
with support from the Rhode Island Board of Governors for Higher 
Education Fund for Excellence. A new FIPSE grant funded the 
development of ChemScholar and MathScholar, two PC-based pro- 
grams with a more powerful and sophisticated graphical user interface. 
These programs will actively engage the student in the direct 
manipulation of scientific objects, concepts, and problem-solving 
processes. FIPSE is also supporting a project entitled Students 
Learning and Teachers Working Using Networked Computer Tech- 
logy. 
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Available Information 



The Chem-Tutor software system for VAX/VMS, including the 
Instructor Expert program, is available for $595. User guides may be 
purchased separately for a nominal fee. Address requests to: 



James L. Fasching 
Department of Chemistry 
Pastore Laboratory 
University of Rhode Island 
Kingston, RI 02881 
401-792-2298 
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Two or three years ago, this category would surely have included 
more projects. The particular three years covered by this volume 
represent a trough between two waves of change in general education. 
In the mid-eighties, many campuses restored structured general edu- 
cation programs that had bt~n eliminated or attenuated by reforms of 
the early 1970's. After a brief lessening of interest in general edu- 
cation, the matter has returned to the forefront of higher education 
concern as institutions make programs more multicultural, interna- 
tionalize the curriculum and develop specific courses required of all 
students. These projects are just now being completed and reporting 
their results. 

The number of general education projects featured here is further 
constrained by the difficulty of evaluating the results of such pro- 
grams. The evaluation methodology is not sufficiently developed, nor 
are the effects of genera) education curricula, other than skills courses 
like English composition, readily discernible in the short run. 

The two projects described below represent quite different aspects 
of general education. Proceeding from a desire to reduce the frag- 
mentation of learning inherent in a curriculum organized in terms of 
distribution requirements, the University of North Texas has intro- 
duced as an option a highly structured course focused on classic texts 
of Western civilization. SUNY/Buffalo's program attacks the problem 
of developing students' general intellectual skills through a freshman/ 
sophomore level course in "Methods of Inquiry." 



UNIVERSITY OF NORTH TEXAS 



The Classic Learning Core 

Purpose 

The Classic Learning Core (CLC) was developed to overcome the 
fragmentation of learning characteristic of the distribution require- 
ments that make up general education curricula. The program con- 
sists of a set of 21 courses, of which students must choose 17, offered 
in nine departments. 

Students must take the courses in a set order, beginning in the 
freshman year and extending through the junior year. A capstone 
course in the senior year rounds off the program. Freshmen take 
English composition and a foreign language and choose among world 
history, classical argument (rhetoric) and art appreciation. Sopho- 
mores take world literature, political science, American history and 
foreign language. Juniors take ethics and psychology. 

The courses are linked by a theme reflecting the classic values of 
"virtue, civL ity and reason." Faculty meet monthly and for two weeks 
during the summer to coordinate syllabi and reading lists so that 
courses can make use of readings and ideas that students have worked 
with or are currently studying in other CLC courses. Faculty 
incorporate these linkages not only in classroom discussion but in 
writing assignments and examinations. 

Approximately 400 students each year elect the Classic Learning 
Core, of whom perhaps two-thirds persist to completion. Students 
who drop out of the Core but remain at the University lose no lime, 
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since all CLC courses satisfy requirements in the regular general 
education program. 

Innovative Features 

The innovation lies in the linkages. The courses are linked not 
only thematically but through the design of syllabi, choice of texts and 
individual items of course content as well. The continuing cross- 
referencing from course to course and the classical humanistic 
approach to course content provide the pedagogical links. 

Students enjoy the advantages of small classes with a minimum of 
lecturing and a maximum of discussion. Insofar as possible students 
are assigned complete primary texts to read. 

Beyond the day-to-day activities of the classroom, students and 
faculty are engaged in common intellectual pursuits. Faculty share 
teaching objectives, and students in the CLC have extended the 
relationship that began in the classroom to publication of a newsletter, 
fund-raising activities and group summer study abroad. 

Evaluation 

The program received formal qualitative evaluations from two 
scholars external to the University, one from within the state of Texas, 
one from outside. They studied curricular materials, sat in on classes 
and interviewed students and faculty. In addition, a member of the 
University faculty conducted an evaluation based on questionnaire 
responses of students and faculty involved in the program. In general, 
evaluations indicate an encouraging degree of satisfaction with the 
program on the part of all groups involved and regardless of the form 
of evaluation. 
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Project Tnroact 



The CLC has provided a point of focus, both academic and social, 
for participating students. It has many of the qualities of a college 
within a much larger university. Faculty who participate regularly in 
the program report a renewal of energy and intellectual enthusiasm, 
much of it resulting from the cross-disciplinary interactions and 
regular association with colleagues in a common pedagogical 
enterprise. 

The CLC has inspired two additional programs. The Classic 
Combination Master of Business Administration Program combines the 
CLC requirements, a liberal arts major, a minor in business and a 
fifth-year masters degree program in business leading to an MBA. 
The Texas Academy of Math and Science permits early college entry 
for students proficient in math and science. All students admitted 
under this program must complete the CLC requirements. 

Unanticipated Prnhlrnns 

Program designers and University administrators had originally 
hoped that the Classic Learning Core would enroll a larger number of 
students, perhaps becoming a required part of the general education 
program for all UNT students. The expense of the program, involving 
as it does small classes and substantial numbers of senior faculty, has 
not permitted expansion beyond the current 400 students per year. 
Furthermore, the number of program drop-outs suggests that the CLC 
would encounter resistance from a substantial number of students 
were it to become a uniform requirement. 

As to the program itself, evaluators noted the need for clearer and 
more frequent connections among courses. This observation has led to 
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more deliberate efforts to take advantage of the thematic and 
curricular coherence of course and program design. 

An advising program under which each CLC student is assigned a 
faculty mentor has not worked as planned, largely because so few 
students take advantage of the opportunities presented. 

Maior Ii isighta a nd Leggong Learned 

The project has demonstrated that a large regional university can 
find faculty interested in expending the physical and intellectual 
energy necessary to offering and sustaining an integrated program of 
humanistic studies and enough students interested in pursuing it to 
justify and reward the effort. Such a program is not, however, likely 
to be suitable for all students or implementable institution-wide 
because of costs and demand on senior faculty teaching time. 

Recognition 

Articles on the Classic Learning Core have appeared in several 
newspapers, including the Chronicle of Higher Education . It has 
received widely-noted favorable mention by both former Secretary of 
Education William Bennett and former Director of the National 
Endowment for the Humanities, Lynne Cheney. 

Project Continuatio n 

University of North Texas administrators have been highly 
supportive of the program since its inception. The CLC is now 
established as a separate unit, fully supported by University funds. An 
endowment has been established and $2 million of a $35 million 
capital campaign have been earmarked to enhance the endowment. 
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Available Information 



Brochures, course syllabi and a student guide to the CLC are 
available by writing to: 

L. Robert Stevens 
Classic Learning Core 
P.O. Box 13827 
University of North Texas 
Denton, TX 76203-5187 
817-565-4672 

or 

J. Don Vann 

Director of Academic Core Programs 
P.O. Box 5187 
University of North Texas 
Denton, TX 76203-5187 
817-565-4030 
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STATE UNIVERSITY OF NEW YORK AT BUFFALO 



Critical Inquiry Studies for the General Undergraduate 

Population 



Purpose 

The project grew from a recognition by many administrators and 
senior faculty that a large number of entering students, though 
successful in their pre-college studies, lack the learning skills necessary 
for success at the college level. Specifically, students have not de- 
veloped the ability to ask questions of a lecture, discussion, or text, to 
approach each analytically and in the particular conceptual modes that 
the discipline they are studying requires. They are accustomed to 
memorizing facts, rather than using them to develop conceptual under- 
standing and to support a series of arguments or claims. 

To address this familiar problem, the project director developed a 
three-credit elective course entitled "Methods of Inquiry" (MOI). MOI 
helps students to learn: 1) strategies for identifying questions critical 
to the discipline they are studymg; 2) different methods for mapping 
course information; 3) principles for monitoring comprehension and 
guiding the effective management of tasks (as opposed to time manage- 
ment), and 4) ways of formulating hypothetical options as a basis for 
reasoned judgment. 

Innova tive Features 

Using as a springboard materials developed for at-risk students by 
Marcia Heiman and Joshua Solomianko, the developers of MOI 
designed a course for individuals at all levels of ability. The course 
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differs from the more familiar small section courses intended to help 
students adjust to the university environment in that it is entirely 
academic in its orientation and is usually offered in sections of 100 or 
more. Thus in its first eight semesters the course enrolled a total of 
2,900 students, filling every seat available. 

Despite the size of the sections, all students receive weekly one-on- 
one assistance in applying MOI strategies to their learning. Lecture 
sections, which meet twice a week, are each supported by a graduate 
teaching assistant and a corps of peer monitors-academically 
successful, former MOI upperclass students trained to serve in this 
role. Monitors, under the supervision of the project director and the 
teaching assistant for the section, meet weekly for 30-45 minutes with 
each student to see how he or she is applying skills covered in the 
lectures and to provide assistance. The academic and personal growth 
of the monitors themselves is a significant by-product of the activity. 

Methods of Inquiry is organized as an independent program under 
the aegis of the Vice Provost for Undergraduate Education. Three 
faculty from different disciplines work with the program each 
semester, along with five teaching assistants and approximately 27 
undergraduate monitors. 

Evaluation 

The project director and her assistants amassed detailed data on 
the results of the first eight semesters of the course (N=2,900 
students). They kept track of improvement in grade-point averages 
and retention figures as weil as student-reported information about 
study practices, attitudes toward learning and regard for the course. 
These data are reported by prior grade-point average, size of class 
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(some small sections were offered), major area of study, ethnic group 
and gender. 

Some major findings include: 

o More than 63% of students improved their GPA's, including 
84% of those with GPA's less than 1.99. The latter group improved 
by an average 1.10. 

o The number of students with GPA's of 3.69 or higher 
doubled. 

o A grade of A in MOI had a high positive correlation with a high 
overall GPA (not including the MOI grade) for the semester in 
which students were enrolled in MOI. 

o Of the undergraduates, MOI students surpassed the overall 
University rate of retention by 11% or more in every admissions 
cohort. 

o Students reported that their concern for good academic 
performance had increased. 

o Students reported that they studied longer outside of class 
and had higher estimations of their own academic strength. 

o Students heavily enrolled in math/science courses are 
significantly less likely to improve GPA's than those enrolled 
predominantly in social sciences and humanities (51% vs. 67%). 
Few statistically significant differences were noted among other 
sub-groups. 
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Project Impact 



"Methods of Inquiry" has become fully institutionalized, enrolling 
about 350 students each semester. Faculty reception of the course has 
been positive. In addition to those who have been directly involved in 
instruction, several faculty have developed discipline-specific materials 
for the course. Related workshops offered by the Office of Teaching 
Effectiveness have led faculty members to use Methods of Inquiry 
materials in their own courses. In particular, the SUNY/Buffalo Den- 
tal and Medical Schools and College of Engineering have all adopted 
programs based on the MOI materials to serve various segments of 
their populations. 

Faculty and administrators from approximately 200 institutions 
have requested and received information about the course. 

Ilnnntir. jpated Problems 

Efforts to involve faculty in two-day training sessions providing 
hands-on experience with course materials have not proven successful. 
Responsiveness to shorter sessions and discipline-related sessions, 
however, has been excellent. (120 faculty attended the last two-hour 
workshop.) 

The considerable difference in results for students in math/science 
and humanities/social science programs has persisted, with math/ 
science students remaining less likely to apply the strategies for 
critical thinking and discipline mastery taught in the course. 

Although student response to the course is positive (72% would 
recommend it to a friend), a consistent 13% have a negative response. 
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This group is not demographically distinguishable from those who 
respond positively. 

Major Insights an d Lessons Learned 

The project demonstrates the possibility of engaging substantial 
numbers of students in large, heterogeneous classes in improving their 
thinking and learning processes. Considerable success appears to be 
attainable by students from diverse backgrounds and achievement 
levels. 

One key to success may be to approach the critical thinking process 
in a non-adversarial way, with an emphasis on learning as a means for 
achieving understanding. This approach encourages independent 
thought and conceptualization. 

Considerable faculty support can be generated for this sort of 
activity as long as it has clear academic substance. Many faculty are 
willing to include in their courses units that teach awareness of critical 
thinking. 

The program would appear to be adaptable to many kinds of 
institutions, but smaller ones may have difficulty with the kind of 
complex organization demanded by the project and may find a 
discipline-related approach more viable. 

Project Continuation 

The Methods of Inquiry Program has been fully funded by the 
University. It serves approximately 700 students a year. 
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Ajvailahle Information 



The project director welcomes telephone inquiries and will make 
available a comprehensive summary of data collection instruments, 
data and working papers. 



Susan R. Schapiro, Director 
Methods of Inquiry Program 
Ellicott Complex 
SUNY at Buffalo 
Buffalo, NY 14261-0012 
716-645-3448 
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TEACHER EDUCATION 



Responding to a wave of national attention to teacher training 
sparked by the Holmes Group and the National Center on Education 
and the Economy (formerly the Carnegie Forum), FIPSE provided 
funding in the late 1980's for some 45 projects in teacher education. 
These projects had such purposes as recruiting minorities and new 
populations to the field, recruiting more academically able and broadly 
prepared people to teaching, and strengthening the subject content of 
pre-service preparation. 

While many of FIPSE's projects have been well implemented, 
demonstration of their success in preparing teachers more effectively 
has proven elusive. Given the fact that by the end of a three-year 
funding period the first graduates of an improved program are just 
entering the first year of teaching, this inconclusiveness is to be 
expected. The Union College program described below is in this 
respect no exception, but it is an unusually innovative and flexible 
program whose progress is exceptionally well documented. The 
Northern Virginia Community College program to train single mothers 
as certified child care providers demonstrates the effectiveness of 
education in alleviating problems of poverty in the community. 

On the other hand, some innovations funded by FIPSE simply did 
not work, though not for lack of care and effort by the institutions 
that undertook them. For example, a set of projects that attempted to 
address the projected teacher shortage by recruiting early retirees (e.g., 
from the military services) and career changers to teaching were 
unable to find the numbers of interested people they thought they 
could. A large percentage of those recruited did not finish the 
program. 
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Other kinds of projects were more susceptible to early demonstra- 
tions of success. The "PreTeacher Assessment Program 11 developed at 
Indiana University of Pennsylvania has shown a high level of corre- 
lation between assessor ratings of pre-service teachers in simulated 
situations and expert judgments of these student teachers' pedagogical 
skills demonstrated in actual classroom settings. 
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INDIANA UNIVERSITY OF PENNSYLVANIA 



The PreTeacher Assessment Center 



Purpose 

During World War II, Dr. Henry Murray, a Harvard psychologist, 
was asked by the Office of Strategic Services to develop a way of 
determining the aptitudes of potential spies for the Allies. Murray and 
a team of psychologists first identified the skills and characteristics 
that were required of espionage agents, such as problem analysis, 
initiative, tenacity, and stress tolerance. Then they tried to match 
these abilities as closely as possible with simulated activities that 
would trigger a demonstration of the kind of behavior required by 
espionage work. The operation was a success~the Allies lost fewer 
agents behind enemy lines, and the quality of the information gathered 
by these agents improved. 

In the post-war years, Murray's ideas were applied to management, 
and in the 1970's, to school administration (the Principal Assessment 
Center). When Indiana University of Pennsylvania, Millersville 
University, Slippery Rock University and Development Dimensions 
International joined together to form the PreTeacher Assessment 
Center, they too followed the approach first developed by Murray and 
his team. 

To assess the abilities of future teachers, the Center first identified 
thirteen basic teaching traits and skills: planning and organization, 
monitoring, leadership, sensitivity, problem analysis, strategic decision 
making, tactical decision making, oral communication, oral presenta- 
tion, written communication, innovativeness, tolerance for stress, and 
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initiative. The staff then designed four simulations that enable 
students to demonstrate these skills as they solve problems and carry 
out tasks. 

Based on the writings, of teachers and faculty at all instructional 
levels, two of the simulations focus on classroom situations (students 
are asked to react to classroom scenes on a videotape and write a 
lesson plan) while the other two simulations assess generic traits such 
as initiative, sensitivity and decision making. The content validity of 
the four assessment simulations was established by three panels of 
teachers, faculty and administrators, whose comments and suggestions 
were incorporated into the simulations. 

The Center also developed some assessor training manuals and 
teacher training modules. Each of the latter consists of a videotape 
that models good teaching and a set of assignments that allow the 
students to practice correct behaviors. The topics of the modules are: 
Beliefs About Teaching, The Sensitive Teacher, The Teacher Leader, 
The Innovative Teacher, and Planning and Organizing. 

Evaluation 

To determine how well the Center's assessment of students predicts 
their teaching behavior, Center staff assessed five students during the 
first week of their student teaching experience. Then the student's 
university supervisor observed the student's teaching, using clinical 
supervisory techniques. The final rating of the student for six of the 
thirteen teaching skills was based on a number of classroom observa- 
tions, on responses to a series of statements designed to make students 
reflect on their teaching, and on the students' journals, lesson plans, 
and other materials. 
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There was remarkable agreement between the Center's assessment 
and the ratings by the university supervisors. On 93% of the occa- 
sions, on-site observations were within a one-point range (on a scale of 
1 to 5) of the assessment results. The researchers concluded that the 
Center's assessment scores accurately predict on-the-job performance 
in six skill categories. 

Project Impact 

The PreTeacher Assessment Center has become self-supporting, and 
is being used by Indiana University of Pennsylvania and Slippery Rock 
University. Between 1987 and 1990 the Center trained 120 assessors 
and assessed 200 students. The Center receives frequent requests for 
information and training from schools and postsecondary institutions. 

M qjor Insights and Lessons Learned 

The Center has assessed students majoring in elementary, secon- 
dary, and special education. The major does not seem to affect 
assessment results, probably because the simulations focus on 
pedagogical skills rather than on knowledge of a certain subject (the 
training modules strongly emphasize the importance of content know- 
ledge). 

In general, students' scores indicated little skill in leadership, 
initiative and innovativeness. Scores in sensitivity, oral presentation, 
problem analysis and decision making were judged barely adequate. 
Students obtained higher scores in written communication, tolerance 
for stress, and oral communication. However, although their oral 
skills were adequate, their presentations lacked enthusiasm and logical 
structure. 
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Project Continuation 



The Center is supporting itself and continues to expand. In 1991- 
92, 100 sophomores were assessed. Carlow College, which has estab- 
lished its own assessment center, trained 25 assessors, and requires 
all education majors to undergo assessment. The Pittsburgh public 
schools are currently piloting the Center's training films, and other 
institutions are considering forming a similar center on their campus. 

Recognition 

The PreTeacher Assessment Center received the 1989 Christa 
McAuliffe Showcase for Excellence Award, 

Available Information 

Copies of articles about the Center, training tapes and other 
materials may be obtained from: 

Rob2rt E. Millward 

Assessment Center 

Indiana University of Pennsylvania 

136 Stouffer Hall 

Indiana, PA 15705-1087 

412-357-5593 



172 



NORTHERN VIRGINIA COMMUNITY COLLEGE 



Child Care Certification for Teen Single Mothers 



Purpose 



The nation is witnessing dramatic increases in early maternity. 
The city of Alexandria has the third highest incidence of teen 
pregnancy in Virginia. Northern Virginia Community College, the 
Alexandria Public Schools and the Alexandria community joined to- 
gether in a program to provide college certificates to young single 
mothers, hoping to break the cycle of poverty through education and 
employable skills. The project's goal was to enable these mothers to 
join the pool of qualified child care professionals in the region through 
a program of study in child development. 

Innovati ve Features 

While the early maternity population is large, few postsecondary 
programs target this group, despite city and college support services. 
In this project Pell Grants were the major source of tuition funding, 
and the city granted transportation, housing, day care and food sub- 
sidy assistance. Students selected for the program were 17 years or 
older, high school graduates with only one child and free of any drug 
dependency or criminal record. By qualifying these young mothers as 
child care professionals, project staff hoped that their economic self- 
sufficiency and child care needs would be resolved simultaneously at 
the work place. 
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Additional community organizations joined in support of the 
program: 

o Alexandria's Mayor, City Manager, and the Director of Social 
Services joined College, community, and public school 
administrators served on the program's advisory committee; 

o Arlington County proviaed tuition funding; 

o the Winkler Foundation funded an apartment unit, furnished 
by community contributions, for several participants; 

o Perpetual Bank opened savings accounts for each student, and 
assisted them in budget management; 

o local physicians provided hygiene and free medical treatment; 

o Alexandria Public Schools guaranteed employment to all students 
completing the certificate program as instructional aides; 

o 15 of the 23 Virginia community colleges, several community 
colleges from Maryland and Pennsylvania, and 10 four-year colleges 
supported the program's dissemination conference; and, 

o program staff noted an increase in college enrollment by parents, 
relatives, and siblings of participating students. 

Evaluation 

Over the three-year period, 138 potential participants were 
interviewed. Of the 55 that enrolled in the program, 40 are now 
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employed and meet the Virginia certification standard to work full 
time as instructional aides in child care settings. The remaining 15 
are currently enrolled at the College in the Associate Degree Program. 
Significantly, students who remained in the program had no second 
pregnancies, and none went on public assistance, other than Section 
Eight Housing. The program is now attracting the sisters of the 
successful participants. 

Unantici pated Prohlpms 

Student retention was the overriding problem faced by the 
program. Even with comprehensive academic and social services, 
personal, health, and financial difficulties caused a few students to 
drop out when the program first started. Second pregnancies were not 
an obstacle, however, nor were academic inadequacies. 

These impoverished young mothers needed to earn money in the 
time they were devoting to schoolwork, and they did not see how they 
would earn enough income working as child care professionals. Thus, 
project staff found it necessary to broaden academic choices and trans- 
fer options for the students. Nevertheless, education alone could not 
overcome the numerous problems faced by these single parents. 

Project Continuation 

The program remains a cooperative venture between the College, 
the city, the school system and the Alexandria community. The 
College continues funding faculty and staff salaries associated with the 
project. When the FIPSE grant ended, the city of Alexandria con- 
tinued support of the staff through January 1991. The City Council 
supported a plan to fund non-teaching personnel for the 1991-1992 
academic year. Arlington County continues to pay tuition for 
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participants, while the city provides free day care for their children. 



The partnership agreement between the College, the city, the school 
system, and the community was the essential ingredient for the pro- 
gram's success. It became increasingly clear that the students faced 
multiple life problems that required multiple social services beyond 
those the College could provide. 

Recognition 

The project received statewide attention, as did the project director 
herself. The project had television coverage nationally (CBS's "Forty 
Hours' 1 ); on public television (WETA); on Washington, D.C. television 
("Capitol Edition"); and on local public television in Virginia. For her 
innovative work the project director won the Washingtonian of the 
Year Award, the Urban League Award, the Boy's Club Award, the 
Hopkins House Award, was described in Jet Magazine , and featured on 
the front cover of Old Dominion Magazine . 

Available Information 

Northern Virginia Community College will make available copies of 
the final report and project handouts. The project director had vol- 
unteered to visit campuses and speak about the program. 

Requests should be addressed to: 

Eula M. Miller 

Education Program Head 

Northern Virginia Community College 

Alexandria, VA 

703-845-6224 
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UNION COLLEGE 



Introducing Teacher Education to a Liberal Arts Campus: 
A Model for Integration with the Disciplines 



Purpose 

Pre-professional programs at liberal arts colleges are frequently 
viewed as marginal to the institution's mission. This is especially true 
of teacher education programs, which seem to attract more than their 
share of suspicion and even outright hostility on the part of liberal 
arts faculty. 

When Union College faculty and administrators set about planning 
the Educational Studies program, the need to integrate it into the 
liberal arts curriculum and to allay faculty apprehensions about 
"creeping pre-professionalism" was paramount in their minds. The 
resulting model of teacher education for a liberal arts campus 
endeavored to attract high ability students to teaching; to integrate 
the study of pedagogy into the liberal arts curriculum; to involve both 
faculty and secondary teachers in the design and teaching of the 
program; and to operate in a cost efficient manner. 

Innova tive Features 

From the beginning, project planners decided not to create a small 
Education Department that would quickly become isolated in Union 
College's staunchly liberal arts campus. Instead, faculty from English, 
languages, mathematics, sciences and social sciences teamed up with 
secondary school teachers to create the Curriculum and Methods 
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courses for these areas. The remainder of the faculty were asked to 
review their departmental curricula from the perspective of teacher 
preparation. They responded by enhancing the scope, rigor and co- 
herence of the major, modifying requirements, eliminating gaps, and 
setting guidelines for acceptance into the Educational Studies program. 
(Before a student can be admitted to the program, he or she must 
obtain a departmental recommendation.) 

To further ensure faculty commitment, individuals teaching 
Educational Studies were given a certain amount of discretion in the 
calculation of their teaching loads, and published articles related to 
pedagogy within a discipline were included in tenure and merit 
reviews. In an effort to raise the consciousness of the entire com- 
munity regarding the importance of pedagogy, the College invited 
outside consultants and held special colloquia entitled "The Science 
And Art Of Teaching." The work of the individual departments is 
coordinated by a Director of Educational Studies, who holds a 12- 
month faculty appointment, performs some administrative duties, and 
reports directly to the Dean of the Faculty. 

The project developed three options: an undergraduate program 
combining education courses with a regular academic major and 
yielding a BS/BA plus certification; a one-year graduate program 
leading to an MAT. degree and certification; and a combined-degree 
program consisting of five years of study and yielding a BS/BA, an 
M.A.T. and certification. The graduate curricula include a full-year, 
half-day teaching internship. 

Educational Studies receives institutional funds through the regular 
academic budget and tuition income from the Office of Graduate and 
C'.itinuing Studies. Thus, the College is able to offer the M.A.T. at 
low cost and to waive some tuition for undergraduates. 
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Evaluation 



Efforts included collection of admissions data, pre- and post- 
surveys, portfolios, and interviews with faculty and teachers as well as 
with students at various points in their curricula and in their first 
year of teaching. 

Although it is too early to judge the quality or retention rates of 
the new teachers, it has already become apparent that the program 
has succeeded in attracting liberal arts students of high caliber to 
teaching. In addition, unlike most education programs, which are 
made up of a majority of humanities majors, Union's Educational 
Studies obtains roughly half of its students from science and math- 
matics. 

As beginning teachers, Union College graduates feel confident and 
well prepared, and are less concerned with salary than with taking on 
new responsibilities, becoming leaders in the schools, and continuing to 
develop professionally. 

Project Impact 

Not only did the faculty who were directly involved in the cur- 
riculum development teams find the experience professionally stimula- 
ting, but their colleagues in departments across the campus became 
aware of the paramount importance of proper pedagogy. As faculty 
collaborated with high school teachers, they developed a new legard 
for these colleagues and a commitment to encourage outstanding stu- 
dents to join and consider teaching as a profession. 
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Unanticipated Problem s 

Although the extensive involvement of liberal arts faculty con- 
stitutes the program's main strength, it also implies a number of 
problems, mainly related to faculty reluctance to engage in team 
teaching with high school teachers during the summers and to offer 
discipline-based courses for the M.A.T. 

Turning teachers into mentors for Union student interns has also 
brought its share of problems. Without special funding, most mentors 
have no incentive to attend summer training sessions, and even with 
the benefit of training, not all mentors are able to work adequately 
with their interns. 

Initially, it was expected that school districts would pay their 
interns. That has proved unworkable, due to objections from teacher 
unions, budget difficulties, and an ample supply of teachers in the 
area. However, interns do receive substitute wages when the mentor 
is absent. 

Major Insig hts an d Lessons Learned 

This project h^s demonstrated the wisdom of involving liberal arts 
departments in the development and instruction of teacher education 
curricula, as opposed to segregating the study of pedagogy in a small 
enclave of its own. Involvement of faculty across the board has also 
ensured that the program will remain an integral part of the College, 
as proven by its having already survived a number of significant 
personnel changes. 

Bringing together faculty and teachers to develop the curriculum 
was an unquestionable strength of the project: it not only made faculty 
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appreciative of their colleagues in the schools but fostered discussion 
of pedagogy and committed the College to collaboration with the 
schools, in efforts such as developing a comprehensive program for at- 
risk students with the Schenectady City Schools. 

Recognition 

The project has been featured at the conference of New York's 
Confederated Organizations of Teacher Education, and showcased by 
the New York State Education Department in statewide meetings of 
teacher education schools. Project personnel have made presentations 
and served as consultants with a number of organizations and institu- 
tions. 

Project Continuation 

All aspects of the project-budget, faculty and administration-have 
been fully institutionalized. 

Available Information 

Program descriptions and other materials are available from: 

Joan M. Krejci 

U.S. Department of Education 
Fund for the Improvement of 

Postsecondary Education 
7th & D Streets, SW Room 3100 
Washington, DC 20202-5175 
202-708-5750 
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ETHICS 

Public concern about the failure of colleges and universities to 
develop a strong sense of ethical behavior in their graduates dates 
from the Watergate era. It has only been strengthened over the years 
by a series of major scandals involving people whose higher education, 
the public feels, should have provided them with a stronger ethical 
rudder. While higher education recognizes the limitations of instruc- 
tion in ethics in channeling behavior, many programs, particularly 
professional ones, have responded seriously to the public concern. 

The St. Cloud State program is a fine example, because of both its 
disciplinary breadth and its use of practicing professionals in the 
classroom. While the ultimate test of such a program, the lifelong 
attitudes and behaviors of the students who have completed the 
courses, can probably never be measured, changes in attitude and 
understanding between the beginning and the end of the course are 
firmly established. 

The quality of ethical judgements in medical treatment has received 
similar public attention and led to the mandatory establishment of 
hospital ethics committees in most states. Higher education has an 
important role to play in training committee members in ethical think- 
ing, a challenge that has been taken up successfully at the national 
level by humanities faculty at the University of Connecticut Medical 
School. 

Both of these projects exemplify a new interest and willingness 
among philosophers to help people learn to deal thoughtfully with the 
practical problems of acting ethically. 
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UNIVERSITY OF CONNECTICUT 

Improving Hospital Ethics Committees (HECs): 
Educating Across the Health Professions 

Purpose 

Concern that further expansion of the scope and complexity of 
modern medicine might soon overwhelm decision making about pa- 
tients has led to the widespread introduction of Hospital Ethics 
Committees (HECs). The membership of such committees is generally 
made up of doctors, nurses, lawyers, hospital administrators, risk 
managers, social workers, philosophers, and clergy, with the medical 
professionals predominating. While the members of these committees 
are on the whole well equipped to understand the technological issues 
associated with complex treatment decisions, few have had any train- 
ing in moral philosophy (ethics), health law, or associated disciplines. 
This project had as its goal the development, piloting and evaluation of 
a week-long ethics and health law curriculum for members of HECs. 

Project Activities 

The curriculum focused on three areas of concern to HECs: 
committee structure and procedures, techniques for case analysis, and 
topical issues. Three sessions were devoted to committee mandate, 
authority and operating practices; six to relevant moral, ethical and 
legal issues and their application; and six to illustrations of their 
application in particular matters that commonly come to HEC 
attention. Ultimately these 15 core sessions were merged into an 
intensive one-week seminar that also included attention to sources of 
literature relevant to core issues and the means of assessing them. 
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The seminar was offered three times between April, 1988 and 
February, 1989 in different parts of the country to groups of about 15 
each, a total of 47 in all. One seminar included only new HEC mem- 
bers; another, members with one to three years' experience; and the 
third, veteran members. Each seminar group was representative of 
the mix of professional backgrounds typical of HECs. All three 
seminars were taught by the co-project directors, project faculty, and 
legal preceptors recruited locally. 

Innovative Feature s 

This is the only program to offer systematic training in bioethical 
and legal issues to HEC members. Given that nearly two-thirds of the 
nation's acute care hospitals have appointed healthcare ethics commit- 
tees, this sort of training is of utmost importance to large numbers of 
people. 

The project directors have further initiated a periodical, Hospital 
Ethics Committee Forum , (now HealthCare Ethics Committee Forum ) 
which appears six times a year. HEC Forum displays the work of 
HEC members and includes HEC members on its editorial advisory 
board. 

Evaluation 

Separation of the original 47 participants into seminars based on 
level of experience allowed a good estimate of the kind of person for 
whom the seminar would be of most value. The primary means of 
assessment was a pre-test/post-test technique in which identical 
questions were posed at the beginning and end of the seminar. The 
test included sections on HEC procedure, formulation of questions that 
should be raised in relation to a particularly challenging hypothetical 
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case, and journals and books that might be consulted in dealing with 
the case. 

The pre-test revealed marked differences among participants with 
differing levels of experience on both the test as a whole and most 
subsections of the test. By the end of the seminar, 73% of the new 
members had improved their score, and the average gain for the group 
was 11.5 points out of 100. By contrast, only 44% of the one- to three- 
year members and 50% of those with longer tenures improved their 
scores. The mean score of the group made up of new members 
improved only slightly, while the mean of the most experienced group 
actually declined. The decline seems to have more to do with chance 
and test conditions than with an actual decline in knowledge, and 
none of the declines was statistically significant. 

These results suggest that the seminar increased knowledge only of 
the new members and particularly in the area of HEC organization 
and functioning. However, participant ratings and comments on the 
seminar as a learning experience were uniformly high for all three 
groups of participants. Indeed, 7 1% of the most experienced group 
gave the seminar "very high' ratings. The sessions on specific analytic 
techniques seem to have been the most appreciated. Thus even the 
more experienced HEC members appear to have gained a good deal, 
even if some of that gain was not measured by the principal assess- 
ment instrument. 

Other than years of HEC experience, the only variable on which 
participant learning and response differed was professional back- 
ground. Even here, while lawyers did notably better and social 
workers somewhat less well than other groups on the post-test, 
members of other professional categories were clustered within a 
seven-point range. Gender, place of employment (hospital or non- 
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hospital), age and committee role (chair or member) resulted in quite 
insignificant differences. 

Pr oject Impact 

The project identified a special subgroup with particular learning 
needs and devised a curriculum and pedagogy to address those needs, 
apparently with measurable success. Beyond the seminars themselves, 
the project has led to the completion of 45 participant papers on HEC 
procedures and bioethical issues, and the founding of HEC Forum for 
HEC members. 

Major Insights and Lessons Learned 

The curriculum appears most successful in bringing new members 
of HECs up to the functioning level of more experienced ones. 
However, participant response suggests that the experience is in- 
tellectually stimulating 3ven for the more veteran participants 

Project Continuation 

State laws and other regulations mandating HECs have resulted in 
an increase in such groups since the project was completed. Building 
on the documented success of this project, the project directors (as co- 
editors of HEC Forum) will disseminate its ethics curriculum nation- 
wide to HECs, supported by a new three-year grant from FIPSE. The 
project includes expanding the numbers of present faculty in health- 
care ethics and health law by conducting one in-service training 
program, and educating members of HECs at 60 sites using the cur- 
riculum modules and materials developed in the initial project. At 
mid-decade the project directors plan to convene a conference of U.S. 
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HEC network leaders who are dedicated to the continuing bioethics/ 
health law education of HECs. 



Available Jpfnrm^tfpTj 

Information about the journal, HealthCare Ethics Committee 
Forum , and about future educational opportunities growing out of the 
project is available from: 



Stuart F. Spicker 

Center for Ethics, Medicine and Public Issues 

Baylor College of Medicine 

1 Baylor Plaza 

Houston, TX 77030 

713-798-3511 
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ST. CLOUD STATE UNIVERSITY 



Responsibility in Professional Life 



Purpose 

The project aimed to increase the ethical sensitivity and social 
responsibility of students in professional programs, and to accomplish 
this purpose within the constraints of curricula that leave little room 
for work in the liberal arts. The project director, then assistant dean 
in the College of Social Sciences, worked with members of the Philos- 
ophy Department and instructors in 25 different courses in 23 depart- 
ments representing each of the University's five colleges. Although 
the project was originally intended to focus on graduate programs 
only, awareness that many professional occupations require only an 
undergraduate degree led to the inclusion of those programs as well. 

Project activities included. I) conducting a ten-week faculty ethics 
seminar to familiarize faculty in professional programs with ethical 
theory and help them learn how to develop case studies; 2) introducing 
faculty-developed course modules on ethical dilemmas in the profes- 
sion, prepared with the advice of practicing professionals, into existing 
courses; 3) developing a graduate seminar in professional ethics open 
to students in all programs. The project was supported by an ethicist 
from the Philosophy Department who worked as a consultant to all 
faculty, and by development of a special resource collection and a 
comprehensive list of ethics centers and programs across the country. 
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Tii^nvqtive Features 



By introducing case-based modules into existing courses, rather 
than creating a new course devoted entirely to ethical issues in the 
field, the project found a more willing audience and had broader 
impact than would have been likely otherwise. The immediacy and 
credibility of the ethical problems examined was enhanced by the 
advice of practicing professionals in the development of course 
materials. These consultants often appeared in case studies presented 
in audio or video format. 

The faculty ethics seminar proved key to the success of the project, 
especially when the original focus on acquainting faculty with ethical 
theory was shifted to an emphasis on the practical problems of prepar- 
ing and presenting case studies in professional ethics in the classroom. 
Movement back and forth between theoretical and pedagogical issues 
over the ten weeks of the seminar also proved effective. Both this 
seminar and the omnidisciplinary graduate course conveyed a sense of 
the commonality of many ethical issues across professions, thus 
helping to break down some of the separateness that characterizes 
programs at the University. 

Evaluation 

The quality of the ethics courses and modules created was assessed 
through evaluations completed by participants at the end of each stage 
of the project. The director further gathered informal feedback from 
people who had close contact with the activities at various stages: the 
professional practitioners used to help develop realistic case study 
materials; the "floating" faculty member from the Philosophy Depart- 
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ment, who worked with all faculty; and the faculty members them- 
selves, especially when assembled for an early summer retreat to 
discuss the past year's activities. The effect of the modules on 
students' ethical sensitivities was assessed by pre-post course 
administration of the nationally-normed Defining Issues Test. 

Project T TP nflct 

Participation of faculty from many fields in a common project 
helped, as such projects frequently do, to break down barriers among 
disciplines. Not only is the social interaction important in reducing 
the sense of separateness, but the discovery of common issues lessens 
the parochial belief in the uniqueness of each field. 

Faculty demonstrated an enhanced awareness of the importance of 
ethical issues in the day-to-day work of practitioners in their fields and 
thus a stronger sense of the importance of introducing ethical ques- 
tions into their courses. This expansion of academic horizons resulted 
in at least half the faculty reporting related new departures in publish- 
ing and professional association work, and conducting workshops for 
disciplinary colleagues. 

The Defining Issues Test revealed statistically significant positive 
changes in student sensitivity to ethical issues as a result of course 
participation, improvement varying directly with the amount of 
exposure to ethics issues in their courses. However, post-course 
scores, even for the graduate seminar, were below the norm for under- 
graduate students nationally. All students showed increases in ethical 
sensitivity, although not always reaching the levels of liberal arts 
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majors. This latter finding suggests a contextual reinforcement pres- 
ent in the liberal arts, but not necessarily present in professional 
disciplines. 

This project has received a good deal of attention from other in- 
stitutions and from ethics centers, with the result that the project 
director has frequently been called on as an informal consultant. 

Unanticip ated Problems 

The Faculty Seminar and resulting case studies were less effective 
in the first year because of an overemphasis on ethical theory. Shift- 
ing to a better balance between theory and case study development re- 
sulted in greater faculty satisfaction and better case studies. The 
realization that no course could make ethicists out of faculty or stu- 
dents resulted in a shift in emphasis toward creating ethical awareness 
and sensitivity. 

Major In sights an d Lessons Learned 

This project reaffirmed some important insights that apply general- 
ly to program development activities in colleges and universities. Fac- 
ulty will respond with commitment and enthusiasm to experiences that 
offer the opportunity for solid professional growth. Mixing faculty (or 
students) from different disciplines in a common academic enterprise 
results in unusually high levels of intellectual stimulation and growth. 
Participants in such projects discover significant commonalities of 
interest and build useful networks. On the other hand, faculty are un- 
comfortable with situations in which they are expected to make expert 
use of knowledge which they have not mastered. Thus it is important 
to emphasize as a principal goal the development of informed aware- 
ness and sensitivity. 
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The involvement of practitioners results in more realistic and 
effective materials and, in this case, revealed a larger number of 
practitioners seriously interested in ethical issues and able to speak 
effectively about them than had been imagined. 



Project Continuation 

Courses and modules developed during the project continue to be 
offered, since those activities were designed to be integrated into the 
curriculum. A second member of the Philosophy Department has 
joined the project to teach the graduate seminar. 



Available Information 



A Partici pant's Handbook provides a detailed account of the way 
the project was conducted, and will be useful for anyone interested in 
establishing a related program. A Listing of Ethics Centers/Programs 
includes academic centers and independent agencies with an interest in 
ethics. These materials may be obtained by writing to: 



David L. Carr, Dean 
Stockton State College 
Jim Leeds Road 

Pomona, New Jersey G8240-9988 
609-652-4512 
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